Apr 12 03:25:49 user nova-compute[72081]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code.
Apr 12 03:25:52 user nova-compute[72081]: DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_linux_bridge.linux_bridge.LinuxBridgePlugin'>' with name 'linux_bridge' {{(pid=72081) initialize /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:44}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_noop.noop.NoOpPlugin'>' with name 'noop' {{(pid=72081) initialize /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:44}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_ovs.ovs.OvsPlugin'>' with name 'ovs' {{(pid=72081) initialize /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:44}}
Apr 12 03:25:52 user nova-compute[72081]: INFO os_vif [-] Loaded VIF plugins: linux_bridge, noop, ovs
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): grep -F node.session.scan /sbin/iscsiadm {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "grep -F node.session.scan /sbin/iscsiadm" returned: 0 in 0.019s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:25:52 user nova-compute[72081]: INFO nova.virt.driver [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] Loading compute driver 'libvirt.LibvirtDriver'
Apr 12 03:25:52 user nova-compute[72081]: INFO nova.compute.provider_config [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] No provider configs found in /etc/nova/provider_config/. If files are present, ensure the Nova process has access.
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] Acquiring lock "singleton_lock" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] Acquired lock "singleton_lock" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] Releasing lock "singleton_lock" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] Full set of CONF: {{(pid=72081) _wait_for_exit_or_signal /usr/local/lib/python3.10/dist-packages/oslo_service/service.py:362}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ******************************************************************************** {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2589}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] Configuration options gathered from: {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2590}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] command line args: ['--config-file', '/etc/nova/nova-cpu.conf'] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2591}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] config files: ['/etc/nova/nova-cpu.conf'] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2592}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ================================================================================ {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2594}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] allow_resize_to_same_host      = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] arq_binding_timeout            = 300 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] backdoor_port                  = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] backdoor_socket                = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] block_device_allocate_retries  = 300 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] block_device_allocate_retries_interval = 5 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cert                           = self.pem {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] compute_driver                 = libvirt.LibvirtDriver {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] compute_monitors               = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] config_dir                     = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] config_drive_format            = iso9660 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] config_file                    = ['/etc/nova/nova-cpu.conf'] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] config_source                  = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] console_host                   = user {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] control_exchange               = nova {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cpu_allocation_ratio           = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] daemon                         = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] debug                          = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] default_access_ip_network_name = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] default_availability_zone      = nova {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] default_ephemeral_format       = ext4 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] default_log_levels             = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] default_schedule_zone          = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] disk_allocation_ratio          = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] enable_new_services            = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] enabled_apis                   = ['osapi_compute'] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] enabled_ssl_apis               = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] flat_injected                  = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] force_config_drive             = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] force_raw_images               = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] graceful_shutdown_timeout      = 5 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] heal_instance_info_cache_interval = 60 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] host                           = user {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] initial_cpu_allocation_ratio   = 4.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] initial_disk_allocation_ratio  = 1.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] initial_ram_allocation_ratio   = 1.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] injected_network_template      = /opt/stack/nova/nova/virt/interfaces.template {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] instance_build_timeout         = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] instance_delete_interval       = 300 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] instance_format                = [instance: %(uuid)s]  {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] instance_name_template         = instance-%08x {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] instance_usage_audit           = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] instance_usage_audit_period    = month {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] instances_path                 = /opt/stack/data/nova/instances {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] internal_service_availability_zone = internal {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] key                            = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] live_migration_retry_count     = 30 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] log_config_append              = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] log_dir                        = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] log_file                       = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] log_options                    = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] log_rotate_interval            = 1 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] log_rotate_interval_type       = days {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] log_rotation_type              = none {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] long_rpc_timeout               = 1800 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] max_concurrent_builds          = 10 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] max_concurrent_live_migrations = 1 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] max_concurrent_snapshots       = 5 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] max_local_block_devices        = 3 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] max_logfile_count              = 30 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] max_logfile_size_mb            = 200 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] maximum_instance_delete_attempts = 5 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] metadata_listen                = 0.0.0.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] metadata_listen_port           = 8775 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] metadata_workers               = 3 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] migrate_max_retries            = -1 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] mkisofs_cmd                    = genisoimage {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] my_block_storage_ip            = 10.0.0.210 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] my_ip                          = 10.0.0.210 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] network_allocate_retries       = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] osapi_compute_listen           = 0.0.0.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] osapi_compute_listen_port      = 8774 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] osapi_compute_unique_server_name_scope =  {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] osapi_compute_workers          = 3 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] password_length                = 12 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] periodic_enable                = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] periodic_fuzzy_delay           = 60 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] pointer_model                  = ps2mouse {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] preallocate_images             = none {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] publish_errors                 = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] pybasedir                      = /opt/stack/nova {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ram_allocation_ratio           = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] rate_limit_burst               = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] rate_limit_except_level        = CRITICAL {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] rate_limit_interval            = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] reboot_timeout                 = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] reclaim_instance_interval      = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] record                         = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] reimage_timeout_per_gb         = 20 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] report_interval                = 10 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] rescue_timeout                 = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] reserved_host_cpus             = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] reserved_host_disk_mb          = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] reserved_host_memory_mb        = 512 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] reserved_huge_pages            = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] resize_confirm_window          = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] resize_fs_using_block_device   = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] resume_guests_state_on_host_boot = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] rootwrap_config                = /etc/nova/rootwrap.conf {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] rpc_response_timeout           = 60 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] run_external_periodic_tasks    = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] running_deleted_instance_action = reap {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] running_deleted_instance_poll_interval = 1800 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] running_deleted_instance_timeout = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] scheduler_instance_sync_interval = 120 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] service_down_time              = 60 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] servicegroup_driver            = db {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] shelved_offload_time           = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] shelved_poll_interval          = 3600 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] shutdown_timeout               = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] source_is_ipv6                 = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ssl_only                       = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] state_path                     = /opt/stack/data/nova {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] sync_power_state_interval      = 600 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] sync_power_state_pool_size     = 1000 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] syslog_log_facility            = LOG_USER {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] tempdir                        = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] timeout_nbd                    = 10 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] transport_url                  = **** {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] update_resources_interval      = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] use_cow_images                 = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] use_eventlog                   = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] use_journal                    = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] use_json                       = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] use_rootwrap_daemon            = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] use_stderr                     = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] use_syslog                     = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vcpu_pin_set                   = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vif_plugging_is_fatal          = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vif_plugging_timeout           = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] virt_mkfs                      = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] volume_usage_poll_interval     = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] watch_log_file                 = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] web                            = /usr/share/spice-html5 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_concurrency.disable_process_locking = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_concurrency.lock_path     = /opt/stack/data/nova {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_metrics.metrics_process_name =  {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api.auth_strategy              = keystone {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api.compute_link_prefix        = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api.dhcp_domain                = novalocal {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api.enable_instance_password   = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api.glance_link_prefix         = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api.instance_list_cells_batch_fixed_size = 100 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api.instance_list_cells_batch_strategy = distributed {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api.instance_list_per_project_cells = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api.list_records_by_skipping_down_cells = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api.local_metadata_per_cell    = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:52 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api.max_limit                  = 1000 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api.metadata_cache_expiration  = 15 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api.neutron_default_tenant_id  = default {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api.use_forwarded_for          = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api.use_neutron_default_nets   = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api.vendordata_dynamic_connect_timeout = 5 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api.vendordata_dynamic_failure_fatal = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api.vendordata_dynamic_read_timeout = 5 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api.vendordata_dynamic_ssl_certfile =  {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api.vendordata_dynamic_targets = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api.vendordata_jsonfile_path   = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api.vendordata_providers       = ['StaticJSON'] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.backend                  = dogpile.cache.memcached {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.backend_argument         = **** {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.config_prefix            = cache.oslo {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.dead_timeout             = 60.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.debug_cache_backend      = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.enable_retry_client      = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.enable_socket_keepalive  = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.enabled                  = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.expiration_time          = 600 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.hashclient_retry_attempts = 2 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.hashclient_retry_delay   = 1.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.memcache_dead_retry      = 300 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.memcache_password        =  {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.memcache_pool_connection_get_timeout = 10 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.memcache_pool_flush_on_reconnect = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.memcache_pool_maxsize    = 10 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.memcache_pool_unused_timeout = 60 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.memcache_sasl_enabled    = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.memcache_servers         = ['localhost:11211'] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.memcache_socket_timeout  = 1.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.memcache_username        =  {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.proxies                  = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.retry_attempts           = 2 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.retry_delay              = 0.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.socket_keepalive_count   = 1 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.socket_keepalive_idle    = 1 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.socket_keepalive_interval = 1 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.tls_allowed_ciphers      = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.tls_cafile               = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.tls_certfile             = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.tls_enabled              = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cache.tls_keyfile              = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cinder.auth_section            = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cinder.auth_type               = password {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cinder.cafile                  = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cinder.catalog_info            = volumev3::publicURL {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cinder.certfile                = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cinder.collect_timing          = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cinder.cross_az_attach         = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cinder.debug                   = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cinder.endpoint_template       = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cinder.http_retries            = 3 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cinder.insecure                = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cinder.keyfile                 = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cinder.os_region_name          = RegionOne {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cinder.split_loggers           = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cinder.timeout                 = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] compute.consecutive_build_service_disable_threshold = 10 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] compute.cpu_dedicated_set      = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] compute.cpu_shared_set         = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] compute.image_type_exclude_list = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] compute.live_migration_wait_for_vif_plug = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] compute.max_concurrent_disk_ops = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] compute.max_disk_devices_to_attach = -1 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] compute.resource_provider_association_refresh = 300 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] compute.shutdown_retry_interval = 10 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] compute.vmdk_allowed_types     = ['streamOptimized', 'monolithicSparse'] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] conductor.workers              = 3 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] console.allowed_origins        = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] console.ssl_ciphers            = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] console.ssl_minimum_version    = default {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] consoleauth.token_ttl          = 600 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cyborg.cafile                  = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cyborg.certfile                = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cyborg.collect_timing          = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cyborg.connect_retries         = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cyborg.connect_retry_delay     = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cyborg.endpoint_override       = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cyborg.insecure                = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cyborg.keyfile                 = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cyborg.max_version             = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cyborg.min_version             = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cyborg.region_name             = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cyborg.service_name            = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cyborg.service_type            = accelerator {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cyborg.split_loggers           = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cyborg.status_code_retries     = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cyborg.status_code_retry_delay = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cyborg.timeout                 = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cyborg.valid_interfaces        = ['internal', 'public'] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] cyborg.version                 = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] database.backend               = sqlalchemy {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] database.connection            = **** {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] database.connection_debug      = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] database.connection_parameters =  {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] database.connection_recycle_time = 3600 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] database.connection_trace      = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] database.db_inc_retry_interval = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] database.db_max_retries        = 20 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] database.db_max_retry_interval = 10 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] database.db_retry_interval     = 1 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] database.max_overflow          = 50 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] database.max_pool_size         = 5 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] database.max_retries           = 10 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] database.mysql_enable_ndb      = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] database.mysql_sql_mode        = TRADITIONAL {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] database.mysql_wsrep_sync_wait = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] database.pool_timeout          = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] database.retry_interval        = 10 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] database.slave_connection      = **** {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] database.sqlite_synchronous    = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api_database.backend           = sqlalchemy {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api_database.connection        = **** {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api_database.connection_debug  = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api_database.connection_parameters =  {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api_database.connection_recycle_time = 3600 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api_database.connection_trace  = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api_database.db_inc_retry_interval = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api_database.db_max_retries    = 20 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api_database.db_max_retry_interval = 10 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api_database.db_retry_interval = 1 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api_database.max_overflow      = 50 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api_database.max_pool_size     = 5 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api_database.max_retries       = 10 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api_database.mysql_enable_ndb  = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api_database.mysql_sql_mode    = TRADITIONAL {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api_database.mysql_wsrep_sync_wait = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api_database.pool_timeout      = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api_database.retry_interval    = 10 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api_database.slave_connection  = **** {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] api_database.sqlite_synchronous = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] devices.enabled_mdev_types     = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ephemeral_storage_encryption.enabled = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ephemeral_storage_encryption.key_size = 512 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.api_servers             = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.cafile                  = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.certfile                = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.collect_timing          = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.connect_retries         = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.connect_retry_delay     = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.debug                   = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.default_trusted_certificate_ids = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.enable_certificate_validation = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.enable_rbd_download     = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.endpoint_override       = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.insecure                = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.keyfile                 = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.max_version             = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.min_version             = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.num_retries             = 3 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.rbd_ceph_conf           =  {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.rbd_connect_timeout     = 5 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.rbd_pool                =  {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.rbd_user                =  {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.region_name             = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.service_name            = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.service_type            = image {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.split_loggers           = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.status_code_retries     = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.status_code_retry_delay = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.timeout                 = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.valid_interfaces        = ['internal', 'public'] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.verify_glance_signatures = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] glance.version                 = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] guestfs.debug                  = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] hyperv.config_drive_cdrom      = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] hyperv.config_drive_inject_password = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] hyperv.dynamic_memory_ratio    = 1.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] hyperv.enable_instance_metrics_collection = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] hyperv.enable_remotefx         = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] hyperv.instances_path_share    =  {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] hyperv.iscsi_initiator_list    = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] hyperv.limit_cpu_features      = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] hyperv.mounted_disk_query_retry_count = 10 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] hyperv.power_state_check_timeframe = 60 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] hyperv.power_state_event_polling_interval = 2 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] hyperv.qemu_img_cmd            = qemu-img.exe {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] hyperv.use_multipath_io        = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] hyperv.volume_attach_retry_count = 10 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] hyperv.volume_attach_retry_interval = 5 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] hyperv.vswitch_name            = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] hyperv.wait_soft_reboot_seconds = 60 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] mks.enabled                    = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] mks.mksproxy_base_url          = http://127.0.0.1:6090/ {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] image_cache.manager_interval   = 2400 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] image_cache.precache_concurrency = 1 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] image_cache.remove_unused_base_images = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] image_cache.subdirectory_name  = _base {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.api_max_retries         = 60 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.api_retry_interval      = 2 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.auth_section            = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.auth_type               = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.cafile                  = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.certfile                = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.collect_timing          = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.connect_retries         = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.connect_retry_delay     = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.endpoint_override       = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.insecure                = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.keyfile                 = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.max_version             = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.min_version             = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.partition_key           = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.peer_list               = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.region_name             = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.serial_console_state_timeout = 10 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.service_name            = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.service_type            = baremetal {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.split_loggers           = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.status_code_retries     = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.status_code_retry_delay = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.timeout                 = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.valid_interfaces        = ['internal', 'public'] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ironic.version                 = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] key_manager.backend            = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] key_manager.fixed_key          = **** {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican.auth_endpoint         = http://localhost/identity/v3 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican.barbican_api_version  = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican.barbican_endpoint     = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican.barbican_endpoint_type = public {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican.barbican_region_name  = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican.cafile                = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican.certfile              = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican.collect_timing        = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican.insecure              = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican.keyfile               = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican.number_of_retries     = 60 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican.retry_delay           = 1 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican.send_service_user_token = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican.split_loggers         = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican.timeout               = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican.verify_ssl            = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican.verify_ssl_path       = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican_service_user.auth_section = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican_service_user.auth_type = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican_service_user.cafile   = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican_service_user.certfile = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican_service_user.collect_timing = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican_service_user.insecure = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican_service_user.keyfile  = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican_service_user.split_loggers = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] barbican_service_user.timeout  = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vault.approle_role_id          = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vault.approle_secret_id        = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vault.cafile                   = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vault.certfile                 = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vault.collect_timing           = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vault.insecure                 = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vault.keyfile                  = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vault.kv_mountpoint            = secret {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vault.kv_version               = 2 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vault.namespace                = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vault.root_token_id            = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vault.split_loggers            = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vault.ssl_ca_crt_file          = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vault.timeout                  = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vault.use_ssl                  = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vault.vault_url                = http://127.0.0.1:8200 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] keystone.cafile                = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] keystone.certfile              = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] keystone.collect_timing        = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] keystone.connect_retries       = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] keystone.connect_retry_delay   = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] keystone.endpoint_override     = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] keystone.insecure              = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] keystone.keyfile               = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] keystone.max_version           = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] keystone.min_version           = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] keystone.region_name           = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] keystone.service_name          = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] keystone.service_type          = identity {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] keystone.split_loggers         = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] keystone.status_code_retries   = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] keystone.status_code_retry_delay = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] keystone.timeout               = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] keystone.valid_interfaces      = ['internal', 'public'] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] keystone.version               = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.connection_uri         =  {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.cpu_mode               = custom {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.cpu_model_extra_flags  = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: WARNING oslo_config.cfg [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] Deprecated: Option "cpu_model" from group "libvirt" is deprecated. Use option "cpu_models" from group "libvirt".
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.cpu_models             = ['Nehalem'] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.cpu_power_governor_high = performance {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.cpu_power_governor_low = powersave {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.cpu_power_management   = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.cpu_power_management_strategy = cpu_state {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.device_detach_attempts = 8 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.device_detach_timeout  = 20 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.disk_cachemodes        = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.disk_prefix            = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.enabled_perf_events    = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.file_backed_memory     = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.gid_maps               = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.hw_disk_discard        = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.hw_machine_type        = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.images_rbd_ceph_conf   =  {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.images_rbd_glance_store_name =  {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.images_rbd_pool        = rbd {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.images_type            = default {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.images_volume_group    = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.inject_key             = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.inject_partition       = -2 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.inject_password        = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.iscsi_iface            = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.iser_use_multipath     = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.live_migration_bandwidth = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.live_migration_completion_timeout = 800 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.live_migration_downtime = 500 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.live_migration_downtime_delay = 75 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.live_migration_downtime_steps = 10 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.live_migration_inbound_addr = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.live_migration_permit_auto_converge = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.live_migration_permit_post_copy = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.live_migration_scheme  = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.live_migration_timeout_action = abort {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.live_migration_tunnelled = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: WARNING oslo_config.cfg [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] Deprecated: Option "live_migration_uri" from group "libvirt" is deprecated for removal (
Apr 12 03:25:53 user nova-compute[72081]: live_migration_uri is deprecated for removal in favor of two other options that
Apr 12 03:25:53 user nova-compute[72081]: allow to change live migration scheme and target URI: ``live_migration_scheme``
Apr 12 03:25:53 user nova-compute[72081]: and ``live_migration_inbound_addr`` respectively.
Apr 12 03:25:53 user nova-compute[72081]: ).  Its value may be silently ignored in the future.
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.live_migration_uri     = qemu+ssh://stack@%s/system {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.live_migration_with_native_tls = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.max_queues             = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.mem_stats_period_seconds = 10 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.nfs_mount_options      = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.nfs_mount_point_base   = /opt/stack/data/nova/mnt {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.num_aoe_discover_tries = 3 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.num_iser_scan_tries    = 5 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.num_memory_encrypted_guests = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.num_nvme_discover_tries = 5 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.num_pcie_ports         = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.num_volume_scan_tries  = 5 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.pmem_namespaces        = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.quobyte_client_cfg     = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.rbd_connect_timeout    = 5 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.rbd_destroy_volume_retries = 12 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.rbd_secret_uuid        = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.rbd_user               = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.realtime_scheduler_priority = 1 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.remote_filesystem_transport = ssh {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.rescue_image_id        = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.rescue_kernel_id       = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.rescue_ramdisk_id      = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.rng_dev_path           = /dev/urandom {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.rx_queue_size          = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.smbfs_mount_options    =  {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.snapshot_compression   = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.snapshot_image_format  = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.snapshots_directory    = /opt/stack/data/nova/instances/snapshots {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.sparse_logical_volumes = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.swtpm_enabled          = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.swtpm_group            = tss {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.swtpm_user             = tss {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.sysinfo_serial         = unique {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.tx_queue_size          = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.uid_maps               = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.use_virtio_for_bridges = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.virt_type              = kvm {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.volume_clear           = zero {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.volume_clear_size      = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.volume_use_multipath   = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.vzstorage_cache_path   = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.vzstorage_log_path     = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.vzstorage_mount_group  = qemu {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.vzstorage_mount_opts   = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.vzstorage_mount_perms  = 0770 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.vzstorage_mount_user   = stack {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] libvirt.wait_soft_reboot_seconds = 120 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.auth_section           = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.auth_type              = password {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.cafile                 = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.certfile               = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.collect_timing         = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.connect_retries        = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.connect_retry_delay    = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.default_floating_pool  = public {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.endpoint_override      = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.extension_sync_interval = 600 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.http_retries           = 3 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.insecure               = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.keyfile                = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.max_version            = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.metadata_proxy_shared_secret = **** {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.min_version            = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.ovs_bridge             = br-int {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.physnets               = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.region_name            = RegionOne {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.service_metadata_proxy = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.service_name           = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.service_type           = network {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.split_loggers          = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.status_code_retries    = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.status_code_retry_delay = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.timeout                = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.valid_interfaces       = ['internal', 'public'] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] neutron.version                = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] notifications.bdms_in_notifications = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] notifications.default_level    = INFO {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] notifications.notification_format = unversioned {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] notifications.notify_on_state_change = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] pci.alias                      = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] pci.device_spec                = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] pci.report_in_placement        = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.auth_section         = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.auth_type            = password {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.auth_url             = http://10.0.0.210/identity {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.cafile               = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.certfile             = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.collect_timing       = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.connect_retries      = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.connect_retry_delay  = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.default_domain_id    = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.default_domain_name  = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.domain_id            = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.domain_name          = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.endpoint_override    = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.insecure             = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.keyfile              = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.max_version          = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.min_version          = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.password             = **** {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.project_domain_id    = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.project_domain_name  = Default {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.project_id           = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.project_name         = service {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.region_name          = RegionOne {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.service_name         = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.service_type         = placement {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.split_loggers        = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.status_code_retries  = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.status_code_retry_delay = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.system_scope         = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.timeout              = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.trust_id             = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.user_domain_id       = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.user_domain_name     = Default {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.user_id              = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.username             = placement {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.valid_interfaces     = ['internal', 'public'] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] placement.version              = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] quota.cores                    = 20 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] quota.count_usage_from_placement = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] quota.driver                   = nova.quota.DbQuotaDriver {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] quota.injected_file_content_bytes = 10240 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] quota.injected_file_path_length = 255 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] quota.injected_files           = 5 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] quota.instances                = 10 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] quota.key_pairs                = 100 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] quota.metadata_items           = 128 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] quota.ram                      = 51200 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] quota.recheck_quota            = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] quota.server_group_members     = 10 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] quota.server_groups            = 10 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] rdp.enabled                    = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] rdp.html5_proxy_base_url       = http://127.0.0.1:6083/ {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] scheduler.enable_isolated_aggregate_filtering = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] scheduler.image_metadata_prefilter = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] scheduler.max_attempts         = 3 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] scheduler.max_placement_results = 1000 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] scheduler.placement_aggregate_required_for_tenants = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] scheduler.query_placement_for_availability_zone = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] scheduler.query_placement_for_image_type_support = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] scheduler.workers              = 3 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] filter_scheduler.host_subset_size = 1 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] filter_scheduler.image_properties_default_architecture = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] filter_scheduler.isolated_hosts = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] filter_scheduler.isolated_images = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] filter_scheduler.max_instances_per_host = 50 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] filter_scheduler.max_io_ops_per_host = 8 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] filter_scheduler.pci_in_placement = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] filter_scheduler.track_instance_changes = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] metrics.required               = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] metrics.weight_multiplier      = 1.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] metrics.weight_of_unavailable  = -10000.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] metrics.weight_setting         = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] serial_console.base_url        = ws://127.0.0.1:6083/ {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] serial_console.enabled         = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] serial_console.port_range      = 10000:20000 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] serial_console.proxyclient_address = 127.0.0.1 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] serial_console.serialproxy_host = 0.0.0.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] serial_console.serialproxy_port = 6083 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] service_user.auth_section      = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] service_user.auth_type         = password {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] service_user.cafile            = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] service_user.certfile          = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] service_user.collect_timing    = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] service_user.insecure          = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] service_user.keyfile           = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] service_user.send_service_user_token = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] service_user.split_loggers     = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] service_user.timeout           = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] spice.agent_enabled            = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] spice.enabled                  = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] spice.html5proxy_base_url      = http://10.0.0.210:6081/spice_auto.html {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] spice.html5proxy_host          = 0.0.0.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] spice.html5proxy_port          = 6082 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] spice.image_compression        = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] spice.jpeg_compression         = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] spice.playback_compression     = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] spice.server_listen            = 127.0.0.1 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] spice.server_proxyclient_address = 127.0.0.1 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] spice.streaming_mode           = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] spice.zlib_compression         = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] upgrade_levels.baseapi         = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] upgrade_levels.cert            = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] upgrade_levels.compute         = auto {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] upgrade_levels.conductor       = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] upgrade_levels.scheduler       = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vendordata_dynamic_auth.auth_section = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vendordata_dynamic_auth.auth_type = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vendordata_dynamic_auth.cafile = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vendordata_dynamic_auth.certfile = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vendordata_dynamic_auth.collect_timing = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vendordata_dynamic_auth.insecure = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vendordata_dynamic_auth.keyfile = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vendordata_dynamic_auth.split_loggers = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vendordata_dynamic_auth.timeout = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vmware.api_retry_count         = 10 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vmware.ca_file                 = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vmware.cache_prefix            = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vmware.cluster_name            = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vmware.connection_pool_size    = 10 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vmware.console_delay_seconds   = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vmware.datastore_regex         = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vmware.host_ip                 = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vmware.host_password           = **** {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vmware.host_port               = 443 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vmware.host_username           = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vmware.insecure                = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vmware.integration_bridge      = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vmware.maximum_objects         = 100 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vmware.pbm_default_policy      = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vmware.pbm_enabled             = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vmware.pbm_wsdl_location       = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vmware.serial_log_dir          = /opt/vmware/vspc {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vmware.serial_port_proxy_uri   = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vmware.serial_port_service_uri = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vmware.task_poll_interval      = 0.5 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vmware.use_linked_clone        = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vmware.vnc_keymap              = en-us {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vmware.vnc_port                = 5900 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vmware.vnc_port_total          = 10000 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vnc.auth_schemes               = ['none'] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vnc.enabled                    = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vnc.novncproxy_base_url        = http://10.0.0.210:6080/vnc_lite.html {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vnc.novncproxy_host            = 0.0.0.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vnc.novncproxy_port            = 6080 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vnc.server_listen              = 0.0.0.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vnc.server_proxyclient_address = 10.0.0.210 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vnc.vencrypt_ca_certs          = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vnc.vencrypt_client_cert       = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vnc.vencrypt_client_key        = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] workarounds.disable_compute_service_check_for_ffu = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] workarounds.disable_fallback_pcpu_query = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] workarounds.disable_group_policy_check_upcall = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] workarounds.disable_libvirt_livesnapshot = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] workarounds.disable_rootwrap   = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] workarounds.enable_numa_live_migration = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] workarounds.enable_qemu_monitor_announce_self = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] workarounds.handle_virt_lifecycle_events = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] workarounds.libvirt_disable_apic = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] workarounds.never_download_image_if_on_rbd = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] workarounds.qemu_monitor_announce_self_count = 3 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] workarounds.qemu_monitor_announce_self_interval = 1 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] workarounds.skip_cpu_compare_at_startup = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] workarounds.skip_cpu_compare_on_dest = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] workarounds.skip_reserve_in_use_ironic_nodes = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] wsgi.api_paste_config          = /etc/nova/api-paste.ini {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] wsgi.client_socket_timeout     = 900 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] wsgi.default_pool_size         = 1000 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] wsgi.keep_alive                = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] wsgi.max_header_line           = 16384 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] wsgi.secure_proxy_ssl_header   = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] wsgi.ssl_ca_file               = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] wsgi.ssl_cert_file             = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] wsgi.ssl_key_file              = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] wsgi.tcp_keepidle              = 600 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] wsgi.wsgi_log_format           = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] zvm.ca_file                    = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] zvm.cloud_connector_url        = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] zvm.image_tmp_path             = /opt/stack/data/nova/images {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] zvm.reachable_timeout          = 300 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_policy.enforce_new_defaults = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_policy.enforce_scope      = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_policy.policy_default_rule = default {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_policy.policy_file        = policy.yaml {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] profiler.connection_string     = messaging:// {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] profiler.enabled               = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] profiler.es_doc_type           = notification {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] profiler.es_scroll_size        = 10000 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] profiler.es_scroll_time        = 2m {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] profiler.filter_error_trace    = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] profiler.hmac_keys             = SECRET_KEY {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] profiler.sentinel_service_name = mymaster {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] profiler.socket_timeout        = 0.1 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] profiler.trace_sqlalchemy      = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] remote_debug.host              = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] remote_debug.port              = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.ssl      = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.ssl_key_file =  {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_rabbit.ssl_version =  {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_notifications.retry = -1 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_messaging_notifications.transport_url = **** {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_limit.auth_section        = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_limit.auth_type           = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_limit.cafile              = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_limit.certfile            = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_limit.collect_timing      = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_limit.connect_retries     = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_limit.connect_retry_delay = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_limit.endpoint_id         = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_limit.endpoint_override   = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_limit.insecure            = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_limit.keyfile             = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_limit.max_version         = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_limit.min_version         = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_limit.region_name         = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_limit.service_name        = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_limit.service_type        = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_limit.split_loggers       = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_limit.status_code_retries = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_limit.status_code_retry_delay = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_limit.timeout             = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_limit.valid_interfaces    = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_limit.version             = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_reports.file_event_handler = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_reports.file_event_handler_interval = 1 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] oslo_reports.log_dir           = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vif_plug_linux_bridge_privileged.capabilities = [12] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vif_plug_linux_bridge_privileged.group = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vif_plug_linux_bridge_privileged.helper_command = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vif_plug_linux_bridge_privileged.logger_name = oslo_privsep.daemon {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vif_plug_linux_bridge_privileged.thread_pool_size = 12 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vif_plug_linux_bridge_privileged.user = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vif_plug_ovs_privileged.capabilities = [12, 1] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vif_plug_ovs_privileged.group  = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vif_plug_ovs_privileged.helper_command = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vif_plug_ovs_privileged.logger_name = oslo_privsep.daemon {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vif_plug_ovs_privileged.thread_pool_size = 12 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] vif_plug_ovs_privileged.user   = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] os_vif_linux_bridge.flat_interface = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] os_vif_linux_bridge.forward_bridge_interface = ['all'] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] os_vif_linux_bridge.iptables_bottom_regex =  {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] os_vif_linux_bridge.iptables_drop_action = DROP {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] os_vif_linux_bridge.iptables_top_regex =  {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] os_vif_linux_bridge.network_device_mtu = 1500 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] os_vif_linux_bridge.use_ipv6   = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] os_vif_linux_bridge.vlan_interface = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] os_vif_ovs.isolate_vif         = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] os_vif_ovs.network_device_mtu  = 1500 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] os_vif_ovs.ovs_vsctl_timeout   = 120 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] os_vif_ovs.ovsdb_connection    = tcp:127.0.0.1:6640 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] os_vif_ovs.ovsdb_interface     = native {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] os_vif_ovs.per_port_bridge     = False {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] os_brick.lock_path             = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] privsep_osbrick.capabilities   = [21] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] privsep_osbrick.group          = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] privsep_osbrick.helper_command = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] privsep_osbrick.logger_name    = os_brick.privileged {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] privsep_osbrick.thread_pool_size = 12 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] privsep_osbrick.user           = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] nova_sys_admin.capabilities    = [0, 1, 2, 3, 12, 21] {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] nova_sys_admin.group           = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] nova_sys_admin.helper_command  = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] nova_sys_admin.logger_name     = oslo_privsep.daemon {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] nova_sys_admin.thread_pool_size = 12 {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] nova_sys_admin.user            = None {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG oslo_service.service [None req-00e1b32f-9b3d-442b-99e0-6529176cab7a None None] ******************************************************************************** {{(pid=72081) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2613}}
Apr 12 03:25:53 user nova-compute[72081]: INFO nova.service [-] Starting compute node (version 0.0.0)
Apr 12 03:25:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Starting native event thread {{(pid=72081) _init_events /opt/stack/nova/nova/virt/libvirt/host.py:492}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Starting green dispatch thread {{(pid=72081) _init_events /opt/stack/nova/nova/virt/libvirt/host.py:498}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Starting connection event dispatch thread {{(pid=72081) initialize /opt/stack/nova/nova/virt/libvirt/host.py:620}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Connecting to libvirt: qemu:///system {{(pid=72081) _get_new_connection /opt/stack/nova/nova/virt/libvirt/host.py:503}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Registering for lifecycle events <nova.virt.libvirt.host.Host object at 0x7f3fb61b2620> {{(pid=72081) _get_new_connection /opt/stack/nova/nova/virt/libvirt/host.py:509}}
Apr 12 03:25:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Registering for connection events: <nova.virt.libvirt.host.Host object at 0x7f3fb61b2620> {{(pid=72081) _get_new_connection /opt/stack/nova/nova/virt/libvirt/host.py:530}}
Apr 12 03:25:53 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Connection event '1' reason 'None'
Apr 12 03:25:53 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Cannot update service status on host "user" since it is not registered.: nova.exception_Remote.ComputeHostNotFound_Remote: Compute host user could not be found.
Apr 12 03:25:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.mount [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Initialising _HostMountState generation 0 {{(pid=72081) host_up /opt/stack/nova/nova/virt/libvirt/volume/mount.py:130}}
Apr 12 03:26:00 user nova-compute[72081]: INFO nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Libvirt host capabilities <capabilities>
Apr 12 03:26:00 user nova-compute[72081]:   <host>
Apr 12 03:26:00 user nova-compute[72081]:     <uuid>e20c3142-5af9-7467-ecd8-70b2e4a210d6</uuid>
Apr 12 03:26:00 user nova-compute[72081]:     <cpu>
Apr 12 03:26:00 user nova-compute[72081]:       <arch>x86_64</arch>
Apr 12 03:26:00 user nova-compute[72081]:       <model>IvyBridge-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <vendor>Intel</vendor>
Apr 12 03:26:00 user nova-compute[72081]:       <microcode version='1070'/>
Apr 12 03:26:00 user nova-compute[72081]:       <counter name='tsc' frequency='2094952000' scaling='no'/>
Apr 12 03:26:00 user nova-compute[72081]:       <topology sockets='6' dies='1' cores='1' threads='1'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature name='ss'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature name='vmx'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature name='pcid'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature name='osxsave'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature name='hypervisor'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature name='arat'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature name='tsc_adjust'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature name='md-clear'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature name='stibp'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature name='arch-capabilities'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature name='ssbd'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature name='invtsc'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature name='rsba'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature name='skip-l1dfl-vmentry'/>
Apr 12 03:26:00 user nova-compute[72081]:       <pages unit='KiB' size='4'/>
Apr 12 03:26:00 user nova-compute[72081]:       <pages unit='KiB' size='2048'/>
Apr 12 03:26:00 user nova-compute[72081]:     </cpu>
Apr 12 03:26:00 user nova-compute[72081]:     <power_management>
Apr 12 03:26:00 user nova-compute[72081]:       <suspend_mem/>
Apr 12 03:26:00 user nova-compute[72081]:       <suspend_disk/>
Apr 12 03:26:00 user nova-compute[72081]:       <suspend_hybrid/>
Apr 12 03:26:00 user nova-compute[72081]:     </power_management>
Apr 12 03:26:00 user nova-compute[72081]:     <iommu support='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     <migration_features>
Apr 12 03:26:00 user nova-compute[72081]:       <live/>
Apr 12 03:26:00 user nova-compute[72081]:       <uri_transports>
Apr 12 03:26:00 user nova-compute[72081]:         <uri_transport>tcp</uri_transport>
Apr 12 03:26:00 user nova-compute[72081]:         <uri_transport>rdma</uri_transport>
Apr 12 03:26:00 user nova-compute[72081]:       </uri_transports>
Apr 12 03:26:00 user nova-compute[72081]:     </migration_features>
Apr 12 03:26:00 user nova-compute[72081]:     <topology>
Apr 12 03:26:00 user nova-compute[72081]:       <cells num='2'>
Apr 12 03:26:00 user nova-compute[72081]:         <cell id='0'>
Apr 12 03:26:00 user nova-compute[72081]:           <memory unit='KiB'>8152920</memory>
Apr 12 03:26:00 user nova-compute[72081]:           <pages unit='KiB' size='4'>2038230</pages>
Apr 12 03:26:00 user nova-compute[72081]:           <pages unit='KiB' size='2048'>0</pages>
Apr 12 03:26:00 user nova-compute[72081]:           <distances>
Apr 12 03:26:00 user nova-compute[72081]:             <sibling id='0' value='10'/>
Apr 12 03:26:00 user nova-compute[72081]:             <sibling id='1' value='20'/>
Apr 12 03:26:00 user nova-compute[72081]:           </distances>
Apr 12 03:26:00 user nova-compute[72081]:           <cpus num='6'>
Apr 12 03:26:00 user nova-compute[72081]:             <cpu id='0' socket_id='0' die_id='0' core_id='0' siblings='0'/>
Apr 12 03:26:00 user nova-compute[72081]:             <cpu id='1' socket_id='2' die_id='0' core_id='0' siblings='1'/>
Apr 12 03:26:00 user nova-compute[72081]:             <cpu id='2' socket_id='4' die_id='0' core_id='0' siblings='2'/>
Apr 12 03:26:00 user nova-compute[72081]:             <cpu id='3' socket_id='6' die_id='0' core_id='0' siblings='3'/>
Apr 12 03:26:00 user nova-compute[72081]:             <cpu id='4' socket_id='8' die_id='0' core_id='0' siblings='4'/>
Apr 12 03:26:00 user nova-compute[72081]:             <cpu id='5' socket_id='10' die_id='0' core_id='0' siblings='5'/>
Apr 12 03:26:00 user nova-compute[72081]:           </cpus>
Apr 12 03:26:00 user nova-compute[72081]:         </cell>
Apr 12 03:26:00 user nova-compute[72081]:         <cell id='1'>
Apr 12 03:26:00 user nova-compute[72081]:           <memory unit='KiB'>8255068</memory>
Apr 12 03:26:00 user nova-compute[72081]:           <pages unit='KiB' size='4'>2063767</pages>
Apr 12 03:26:00 user nova-compute[72081]:           <pages unit='KiB' size='2048'>0</pages>
Apr 12 03:26:00 user nova-compute[72081]:           <distances>
Apr 12 03:26:00 user nova-compute[72081]:             <sibling id='0' value='20'/>
Apr 12 03:26:00 user nova-compute[72081]:             <sibling id='1' value='10'/>
Apr 12 03:26:00 user nova-compute[72081]:           </distances>
Apr 12 03:26:00 user nova-compute[72081]:           <cpus num='6'>
Apr 12 03:26:00 user nova-compute[72081]:             <cpu id='6' socket_id='12' die_id='0' core_id='0' siblings='6'/>
Apr 12 03:26:00 user nova-compute[72081]:             <cpu id='7' socket_id='14' die_id='0' core_id='0' siblings='7'/>
Apr 12 03:26:00 user nova-compute[72081]:             <cpu id='8' socket_id='16' die_id='0' core_id='0' siblings='8'/>
Apr 12 03:26:00 user nova-compute[72081]:             <cpu id='9' socket_id='18' die_id='0' core_id='0' siblings='9'/>
Apr 12 03:26:00 user nova-compute[72081]:             <cpu id='10' socket_id='20' die_id='0' core_id='0' siblings='10'/>
Apr 12 03:26:00 user nova-compute[72081]:             <cpu id='11' socket_id='22' die_id='0' core_id='0' siblings='11'/>
Apr 12 03:26:00 user nova-compute[72081]:           </cpus>
Apr 12 03:26:00 user nova-compute[72081]:         </cell>
Apr 12 03:26:00 user nova-compute[72081]:       </cells>
Apr 12 03:26:00 user nova-compute[72081]:     </topology>
Apr 12 03:26:00 user nova-compute[72081]:     <cache>
Apr 12 03:26:00 user nova-compute[72081]:       <bank id='0' level='3' type='both' size='15' unit='MiB' cpus='0'/>
Apr 12 03:26:00 user nova-compute[72081]:       <bank id='2' level='3' type='both' size='15' unit='MiB' cpus='1'/>
Apr 12 03:26:00 user nova-compute[72081]:       <bank id='4' level='3' type='both' size='15' unit='MiB' cpus='2'/>
Apr 12 03:26:00 user nova-compute[72081]:       <bank id='6' level='3' type='both' size='15' unit='MiB' cpus='3'/>
Apr 12 03:26:00 user nova-compute[72081]:       <bank id='8' level='3' type='both' size='15' unit='MiB' cpus='4'/>
Apr 12 03:26:00 user nova-compute[72081]:       <bank id='10' level='3' type='both' size='15' unit='MiB' cpus='5'/>
Apr 12 03:26:00 user nova-compute[72081]:       <bank id='12' level='3' type='both' size='15' unit='MiB' cpus='6'/>
Apr 12 03:26:00 user nova-compute[72081]:       <bank id='14' level='3' type='both' size='15' unit='MiB' cpus='7'/>
Apr 12 03:26:00 user nova-compute[72081]:       <bank id='16' level='3' type='both' size='15' unit='MiB' cpus='8'/>
Apr 12 03:26:00 user nova-compute[72081]:       <bank id='18' level='3' type='both' size='15' unit='MiB' cpus='9'/>
Apr 12 03:26:00 user nova-compute[72081]:       <bank id='20' level='3' type='both' size='15' unit='MiB' cpus='10'/>
Apr 12 03:26:00 user nova-compute[72081]:       <bank id='22' level='3' type='both' size='15' unit='MiB' cpus='11'/>
Apr 12 03:26:00 user nova-compute[72081]:     </cache>
Apr 12 03:26:00 user nova-compute[72081]:     <secmodel>
Apr 12 03:26:00 user nova-compute[72081]:       <model>apparmor</model>
Apr 12 03:26:00 user nova-compute[72081]:       <doi>0</doi>
Apr 12 03:26:00 user nova-compute[72081]:     </secmodel>
Apr 12 03:26:00 user nova-compute[72081]:     <secmodel>
Apr 12 03:26:00 user nova-compute[72081]:       <model>dac</model>
Apr 12 03:26:00 user nova-compute[72081]:       <doi>0</doi>
Apr 12 03:26:00 user nova-compute[72081]:       <baselabel type='kvm'>+64055:+108</baselabel>
Apr 12 03:26:00 user nova-compute[72081]:       <baselabel type='qemu'>+64055:+108</baselabel>
Apr 12 03:26:00 user nova-compute[72081]:     </secmodel>
Apr 12 03:26:00 user nova-compute[72081]:   </host>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='alpha'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>64</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-alpha</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>clipper</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='armv6l'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>32</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-arm</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>integratorcp</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>ast2600-evb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>borzoi</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>spitz</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>virt-2.7</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>nuri</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>mcimx7d-sabre</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>romulus-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-3.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-5.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>npcm750-evb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>virt-2.10</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>rainier-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mps3-an547</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>musca-b1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>realview-pbx-a9</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>versatileab</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>kzm</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>virt-2.8</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>musca-a</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-3.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mcimx6ul-evk</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-5.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>smdkc210</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>sx1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>virt-2.11</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>imx25-pdk</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>stm32vldiscovery</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>virt-2.9</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>orangepi-pc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>quanta-q71l-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>z2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-5.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>xilinx-zynq-a9</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>tosa</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mps2-an500</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>virt-2.12</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>mps2-an521</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>sabrelite</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mps2-an511</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>canon-a1100</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>realview-eb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>quanta-gbs-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>emcraft-sf2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>realview-pb-a8</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-4.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>raspi1ap</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>palmetto-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>sx1-v1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>n810</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>g220a-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>n800</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>tacoma-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-4.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>quanta-gsj</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>versatilepb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>terrier</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mainstone</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>realview-eb-mpcore</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>supermicrox11-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-4.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>witherspoon-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>mps3-an524</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1' deprecated='yes'>swift-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>kudo-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>vexpress-a9</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>midway</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>musicpal</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>lm3s811evb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>lm3s6965evb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>microbit</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mps2-an505</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mps2-an385</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-6.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>cubieboard</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>verdex</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>netduino2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mps2-an386</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-6.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>raspi2b</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>vexpress-a15</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>fuji-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-6.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine canonical='virt-6.2' maxCpus='512'>virt</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>sonorapass-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>cheetah</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>virt-2.6</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>ast2500-evb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>highbank</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>akita</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>connex</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>netduinoplus2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>collie</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>raspi0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>fp5280g2-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='armv7l'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>32</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-arm</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>integratorcp</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>ast2600-evb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>borzoi</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>spitz</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>virt-2.7</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>nuri</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>mcimx7d-sabre</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>romulus-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-3.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-5.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>npcm750-evb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>virt-2.10</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>rainier-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mps3-an547</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>musca-b1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>realview-pbx-a9</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>versatileab</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>kzm</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>virt-2.8</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>musca-a</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-3.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mcimx6ul-evk</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-5.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>smdkc210</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>sx1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>virt-2.11</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>imx25-pdk</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>stm32vldiscovery</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>virt-2.9</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>orangepi-pc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>quanta-q71l-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>z2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-5.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>xilinx-zynq-a9</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>tosa</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mps2-an500</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>virt-2.12</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>mps2-an521</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>sabrelite</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mps2-an511</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>canon-a1100</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>realview-eb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>quanta-gbs-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>emcraft-sf2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>realview-pb-a8</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-4.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>raspi1ap</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>palmetto-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>sx1-v1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>n810</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>g220a-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>n800</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>tacoma-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-4.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>quanta-gsj</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>versatilepb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>terrier</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mainstone</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>realview-eb-mpcore</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>supermicrox11-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-4.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>witherspoon-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>mps3-an524</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1' deprecated='yes'>swift-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>kudo-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>vexpress-a9</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>midway</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>musicpal</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>lm3s811evb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>lm3s6965evb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>microbit</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mps2-an505</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mps2-an385</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-6.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>cubieboard</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>verdex</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>netduino2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mps2-an386</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-6.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>raspi2b</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>vexpress-a15</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>fuji-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-6.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine canonical='virt-6.2' maxCpus='512'>virt</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>sonorapass-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>cheetah</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>virt-2.6</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>ast2500-evb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>highbank</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>akita</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>connex</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>netduinoplus2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>collie</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>raspi0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>fp5280g2-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='aarch64'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>64</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-aarch64</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>integratorcp</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>ast2600-evb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>borzoi</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>spitz</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>virt-2.7</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>nuri</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>mcimx7d-sabre</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>romulus-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-3.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-5.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>npcm750-evb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>virt-2.10</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>rainier-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mps3-an547</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>virt-2.8</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>musca-b1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>realview-pbx-a9</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>versatileab</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>kzm</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>musca-a</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-3.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mcimx6ul-evk</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-5.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>smdkc210</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>sx1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>virt-2.11</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>imx25-pdk</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>stm32vldiscovery</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>virt-2.9</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>orangepi-pc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>quanta-q71l-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>z2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-5.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>xilinx-zynq-a9</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='6'>xlnx-zcu102</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>tosa</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mps2-an500</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>virt-2.12</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>mps2-an521</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>sabrelite</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mps2-an511</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>canon-a1100</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>realview-eb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>quanta-gbs-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>emcraft-sf2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>realview-pb-a8</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>sbsa-ref</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-4.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>raspi1ap</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>palmetto-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>sx1-v1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>n810</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>g220a-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>n800</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>tacoma-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-4.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>quanta-gsj</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>versatilepb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>terrier</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mainstone</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>realview-eb-mpcore</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>supermicrox11-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-4.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>witherspoon-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>mps3-an524</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1' deprecated='yes'>swift-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>kudo-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>vexpress-a9</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>midway</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>musicpal</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>lm3s811evb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>lm3s6965evb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>microbit</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mps2-an505</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mps2-an385</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-6.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>raspi3ap</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>cubieboard</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>verdex</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>netduino2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>xlnx-versal-virt</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mps2-an386</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-6.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>raspi3b</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>raspi2b</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>vexpress-a15</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2'>fuji-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='512'>virt-6.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine canonical='virt-6.2' maxCpus='512'>virt</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>sonorapass-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>cheetah</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>virt-2.6</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>ast2500-evb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>highbank</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>akita</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>connex</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>netduinoplus2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>collie</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>raspi0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>fp5280g2-bmc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <acpi default='on' toggle='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='cris'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>32</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-cris</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>axis-dev88</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='i686'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>32</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-i386</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-jammy</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine canonical='pc-i440fx-jammy' maxCpus='255'>ubuntu</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-impish-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-5.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.12</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-xenial</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-6.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine canonical='pc-i440fx-6.2' maxCpus='255'>pc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-4.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.5</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-4.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-focal</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-hirsute</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-q35-xenial</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-jammy-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-5.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-1.5</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-q35-2.7</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-eoan-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-zesty</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-disco-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-groovy</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-groovy</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-artful</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-trusty</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-eoan-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-focal-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-bionic-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-artful</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.7</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-6.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-yakkety</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-q35-2.4</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-cosmic-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-2.10</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>x-remote</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-5.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-1.7</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-2.9</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.11</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-3.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-6.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-4.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-jammy</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine canonical='pc-q35-jammy' maxCpus='288'>ubuntu-q35</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.4</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-4.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-eoan</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-jammy-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-5.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.9</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-bionic-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>isapc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-1.4</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-cosmic</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-q35-2.6</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-3.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-bionic</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-disco-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-cosmic</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-2.12</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-bionic</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-groovy-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-disco</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-cosmic-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-wily</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-impish</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-6.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-impish</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.6</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-impish-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-hirsute</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-4.0.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-hirsute-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-1.6</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-5.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-2.8</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.10</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-3.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-6.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-zesty</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-4.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-focal</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>microvm</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.3</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-focal-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-disco</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-4.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-groovy-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-hirsute-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-5.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-6.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine canonical='pc-q35-6.2' maxCpus='288'>q35</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.8</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-eoan</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-q35-2.5</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-3.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-q35-yakkety</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-2.11</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='kvm'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <pae/>
Apr 12 03:26:00 user nova-compute[72081]:       <nonpae/>
Apr 12 03:26:00 user nova-compute[72081]:       <acpi default='on' toggle='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:       <apic default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='m68k'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>32</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-m68k</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mcf5208evb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>an5206</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>virt-6.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>q800</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>virt-6.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine canonical='virt-6.2' maxCpus='1'>virt</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>next-cube</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>virt-6.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='microblaze'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>32</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-microblaze</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>petalogix-s3adsp1800</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>petalogix-ml605</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>xlnx-zynqmp-pmu</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='microblazeel'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>32</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-microblazeel</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>petalogix-s3adsp1800</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>petalogix-ml605</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>xlnx-zynqmp-pmu</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='mips'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>32</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-mips</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='16'>malta</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mipssim</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='mipsel'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>32</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-mipsel</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='16'>malta</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mipssim</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='mips64'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>64</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-mips64</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='16'>malta</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mipssim</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>pica61</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>magnum</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='mips64el'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>64</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-mips64el</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='16'>malta</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='16'>loongson3-virt</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mipssim</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>pica61</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>magnum</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='16'>boston</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>fuloong2e</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='ppc'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>32</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-ppc</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>g3beige</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>virtex-ml507</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mac99</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='32'>ppce500</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>pegasos2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>sam460ex</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>bamboo</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>40p</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>ref405ep</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='15'>mpc8544ds</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>taihu</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='ppc64'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>64</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-ppc64</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-jammy</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine canonical='pseries-jammy' maxCpus='2147483647'>pseries</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2048'>powernv9</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine canonical='powernv9' maxCpus='2048'>powernv</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>taihu</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-4.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='15'>mpc8544ds</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-6.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.5</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2048'>powernv10</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-xenial</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-4.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-6.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-yakkety</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.6</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='32'>ppce500</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-bionic-sxxm</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.7</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-3.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-5.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>40p</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.8</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>pegasos2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-hirsute</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-3.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-5.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-eoan</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.9</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-zesty</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>bamboo</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-groovy</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-focal</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>g3beige</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-5.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-disco</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.12-sxxm</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.10</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>virtex-ml507</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.11</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-cosmic</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-bionic</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.12</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mac99</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-impish</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-artful</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>sam460ex</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>ref405ep</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.3</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2048'>powernv8</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-4.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-6.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.4</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='ppc64le'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>64</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-ppc64le</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-jammy</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine canonical='pseries-jammy' maxCpus='2147483647'>pseries</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2048'>powernv9</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine canonical='powernv9' maxCpus='2048'>powernv</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>taihu</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-4.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='15'>mpc8544ds</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-6.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.5</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2048'>powernv10</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-xenial</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-4.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-6.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-yakkety</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.6</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='32'>ppce500</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-bionic-sxxm</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.7</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-3.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-5.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>40p</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.8</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>pegasos2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-hirsute</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-3.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-5.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-eoan</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.9</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-zesty</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>bamboo</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-groovy</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-focal</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>g3beige</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-5.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-disco</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.12-sxxm</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.10</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>virtex-ml507</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.11</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-cosmic</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-bionic</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.12</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>mac99</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-impish</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-artful</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>sam460ex</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>ref405ep</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.3</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2048'>powernv8</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-4.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-6.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='2147483647'>pseries-2.4</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='riscv32'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>32</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-riscv32</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='8'>spike</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>opentitan</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='5'>sifive_u</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>sifive_e</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='8'>virt</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='riscv64'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>64</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-riscv64</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='8'>spike</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='5'>microchip-icicle-kit</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='5'>sifive_u</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>shakti_c</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>sifive_e</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='8'>virt</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='s390x'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>64</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-s390x</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-jammy</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine canonical='s390-ccw-virtio-jammy' maxCpus='248'>s390-ccw-virtio</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-4.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-5.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-artful</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-3.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-groovy</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-hirsute</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-disco</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-2.12</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-2.6</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-yakkety</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-eoan</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-2.9</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-6.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-5.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-3.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-4.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-2.5</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-2.11</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-xenial</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-focal</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-2.8</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-impish</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-bionic</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-5.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-6.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-zesty</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-4.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-cosmic</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-2.4</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-2.10</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-2.7</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='248'>s390-ccw-virtio-6.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='sh4'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>32</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-sh4</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>shix</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>r2d</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='sh4eb'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>64</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-sh4eb</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>shix</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>r2d</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='sparc'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>32</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-sparc</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>SS-5</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>SS-20</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>LX</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>SPARCClassic</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>leon3_generic</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>SPARCbook</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>SS-4</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>SS-600MP</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>SS-10</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>Voyager</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='sparc64'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>64</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-sparc64</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>sun4u</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>niagara</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>sun4v</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='x86_64'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>64</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-x86_64</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-jammy</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine canonical='pc-i440fx-jammy' maxCpus='255'>ubuntu</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-impish-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-5.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.12</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-xenial</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-6.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine canonical='pc-i440fx-6.2' maxCpus='255'>pc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-4.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.5</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-4.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-hirsute</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-focal</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-q35-xenial</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-jammy-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-5.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-1.5</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-q35-2.7</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-eoan-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-zesty</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-disco-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-groovy</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-groovy</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-artful</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-trusty</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-focal-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-eoan-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-bionic-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-artful</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.7</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-6.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-yakkety</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-q35-2.4</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-cosmic-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-2.10</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>x-remote</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-5.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-1.7</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-2.9</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.11</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-3.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-6.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-4.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-jammy</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine canonical='pc-q35-jammy' maxCpus='288'>ubuntu-q35</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.4</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-4.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-eoan</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-jammy-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-5.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.9</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-bionic-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='1'>isapc</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-1.4</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-cosmic</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-q35-2.6</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-3.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-bionic</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-disco-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-cosmic</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-2.12</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-bionic</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-groovy-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-disco</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-cosmic-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-wily</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-impish</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.6</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-6.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-impish</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-impish-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-hirsute</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-4.0.1</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-hirsute-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-1.6</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-5.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-2.8</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.10</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-3.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-zesty</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-4.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-focal</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>microvm</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-6.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.3</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-disco</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-focal-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-4.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-groovy-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-hirsute-hpb</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-5.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-2.8</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-6.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine canonical='pc-q35-6.2' maxCpus='288'>q35</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-eoan</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-q35-2.5</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-i440fx-3.0</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='255'>pc-q35-yakkety</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='288'>pc-q35-2.11</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='kvm'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <acpi default='on' toggle='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:       <apic default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='xtensa'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>32</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-xtensa</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>sim</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='32'>kc705</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='32'>ml605</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='32'>ml605-nommu</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='32'>virt</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='32'>lx60-nommu</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='32'>lx200</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='32'>lx200-nommu</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='32'>lx60</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='32'>kc705-nommu</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]:   <guest>
Apr 12 03:26:00 user nova-compute[72081]:     <os_type>hvm</os_type>
Apr 12 03:26:00 user nova-compute[72081]:     <arch name='xtensaeb'>
Apr 12 03:26:00 user nova-compute[72081]:       <wordsize>32</wordsize>
Apr 12 03:26:00 user nova-compute[72081]:       <emulator>/usr/bin/qemu-system-xtensaeb</emulator>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='4'>sim</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='32'>kc705</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='32'>ml605</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='32'>ml605-nommu</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='32'>virt</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='32'>lx60-nommu</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='32'>lx200</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='32'>lx200-nommu</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='32'>lx60</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <machine maxCpus='32'>kc705-nommu</machine>
Apr 12 03:26:00 user nova-compute[72081]:       <domain type='qemu'/>
Apr 12 03:26:00 user nova-compute[72081]:     </arch>
Apr 12 03:26:00 user nova-compute[72081]:     <features>
Apr 12 03:26:00 user nova-compute[72081]:       <cpuselection/>
Apr 12 03:26:00 user nova-compute[72081]:       <deviceboot/>
Apr 12 03:26:00 user nova-compute[72081]:       <disksnapshot default='on' toggle='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     </features>
Apr 12 03:26:00 user nova-compute[72081]:   </guest>
Apr 12 03:26:00 user nova-compute[72081]: </capabilities>
Apr 12 03:26:00 user nova-compute[72081]: 
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for alpha via machine types: {None} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch alpha / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-alpha' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for armv6l via machine types: {'virt', None} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch armv6l / virt_type kvm / machine_type virt: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-arm' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch armv6l / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-arm' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for armv7l via machine types: {'virt'} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch armv7l / virt_type kvm / machine_type virt: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-arm' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for aarch64 via machine types: {'virt'} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch aarch64 / virt_type kvm / machine_type virt: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-aarch64' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for cris via machine types: {None} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch cris / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-cris' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for i686 via machine types: {'pc', 'ubuntu-q35', 'ubuntu', 'q35'} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Libvirt host hypervisor capabilities for arch=i686 and machine_type=pc:
Apr 12 03:26:00 user nova-compute[72081]: <domainCapabilities>
Apr 12 03:26:00 user nova-compute[72081]:   <path>/usr/bin/qemu-system-i386</path>
Apr 12 03:26:00 user nova-compute[72081]:   <domain>kvm</domain>
Apr 12 03:26:00 user nova-compute[72081]:   <machine>pc-i440fx-6.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:   <arch>i686</arch>
Apr 12 03:26:00 user nova-compute[72081]:   <vcpu max='255'/>
Apr 12 03:26:00 user nova-compute[72081]:   <iothreads supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:   <os supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:     <enum name='firmware'/>
Apr 12 03:26:00 user nova-compute[72081]:     <loader supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/OVMF/OVMF_CODE.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/OVMF/OVMF_CODE.secboot.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/AAVMF/AAVMF_CODE.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/AAVMF/AAVMF32_CODE.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/OVMF/OVMF_CODE.ms.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='type'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>rom</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>pflash</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='readonly'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>yes</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>no</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='secure'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>no</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </loader>
Apr 12 03:26:00 user nova-compute[72081]:   </os>
Apr 12 03:26:00 user nova-compute[72081]:   <cpu>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='host-passthrough' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='hostPassthroughMigratable'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>off</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='maximum' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='maximumMigratable'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>on</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>off</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='host-model' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <vendor>Intel</vendor>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ss'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='vmx'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='pcid'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='hypervisor'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='arat'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='tsc_adjust'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='umip'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='md-clear'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='stibp'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='arch-capabilities'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ssbd'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ibpb'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ibrs'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='amd-stibp'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='amd-ssbd'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='disable' name='erms'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='disable' name='xsaveopt'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='disable' name='lm'/>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='custom' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>qemu64</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>qemu32</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>phenom</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>pentium3</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>pentium2</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>pentium</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>n270</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>kvm64</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>kvm32</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>coreduo</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>core2duo</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>athlon</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Westmere-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Westmere</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Snowridge</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Server-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Server</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Client-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Client</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>SandyBridge-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>SandyBridge</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Penryn</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G5</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G4</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G3</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G2</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G1</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Nehalem-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Nehalem</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>IvyBridge-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>IvyBridge</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Icelake-Server-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Icelake-Server</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC-Rome</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC-Milan</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC-IBPB</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Dhyana</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Cooperlake</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Conroe</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Cascadelake-Server</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>486</model>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:   </cpu>
Apr 12 03:26:00 user nova-compute[72081]:   <memoryBacking supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:     <enum name='sourceType'>
Apr 12 03:26:00 user nova-compute[72081]:       <value>file</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>anonymous</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>memfd</value>
Apr 12 03:26:00 user nova-compute[72081]:     </enum>
Apr 12 03:26:00 user nova-compute[72081]:   </memoryBacking>
Apr 12 03:26:00 user nova-compute[72081]:   <devices>
Apr 12 03:26:00 user nova-compute[72081]:     <disk supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='diskDevice'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>disk</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>cdrom</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>floppy</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>lun</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='bus'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>ide</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>fdc</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>scsi</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>usb</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>sata</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='model'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-non-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </disk>
Apr 12 03:26:00 user nova-compute[72081]:     <graphics supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='type'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>sdl</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>vnc</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>spice</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>egl-headless</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </graphics>
Apr 12 03:26:00 user nova-compute[72081]:     <video supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='modelType'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>vga</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>cirrus</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>vmvga</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>qxl</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>none</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>bochs</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>ramfb</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </video>
Apr 12 03:26:00 user nova-compute[72081]:     <hostdev supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='mode'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>subsystem</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='startupPolicy'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>default</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>mandatory</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>requisite</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>optional</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='subsysType'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>usb</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>pci</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>scsi</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='capsType'/>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='pciBackend'/>
Apr 12 03:26:00 user nova-compute[72081]:     </hostdev>
Apr 12 03:26:00 user nova-compute[72081]:     <rng supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='model'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-non-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='backendModel'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>random</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>egd</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>builtin</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </rng>
Apr 12 03:26:00 user nova-compute[72081]:     <filesystem supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='driverType'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>path</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>handle</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtiofs</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </filesystem>
Apr 12 03:26:00 user nova-compute[72081]:     <tpm supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='model'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>tpm-tis</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>tpm-crb</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='backendModel'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>passthrough</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>emulator</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </tpm>
Apr 12 03:26:00 user nova-compute[72081]:   </devices>
Apr 12 03:26:00 user nova-compute[72081]:   <features>
Apr 12 03:26:00 user nova-compute[72081]:     <gic supported='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     <vmcoreinfo supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <genid supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <backingStoreInput supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <backup supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <sev supported='no'/>
Apr 12 03:26:00 user nova-compute[72081]:   </features>
Apr 12 03:26:00 user nova-compute[72081]: </domainCapabilities>
Apr 12 03:26:00 user nova-compute[72081]:  {{(pid=72081) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Libvirt host hypervisor capabilities for arch=i686 and machine_type=ubuntu-q35:
Apr 12 03:26:00 user nova-compute[72081]: <domainCapabilities>
Apr 12 03:26:00 user nova-compute[72081]:   <path>/usr/bin/qemu-system-i386</path>
Apr 12 03:26:00 user nova-compute[72081]:   <domain>kvm</domain>
Apr 12 03:26:00 user nova-compute[72081]:   <machine>pc-q35-jammy</machine>
Apr 12 03:26:00 user nova-compute[72081]:   <arch>i686</arch>
Apr 12 03:26:00 user nova-compute[72081]:   <vcpu max='288'/>
Apr 12 03:26:00 user nova-compute[72081]:   <iothreads supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:   <os supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:     <enum name='firmware'/>
Apr 12 03:26:00 user nova-compute[72081]:     <loader supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/OVMF/OVMF_CODE.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/OVMF/OVMF_CODE.secboot.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/AAVMF/AAVMF_CODE.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/AAVMF/AAVMF32_CODE.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/OVMF/OVMF_CODE.ms.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='type'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>rom</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>pflash</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='readonly'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>yes</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>no</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='secure'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>no</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </loader>
Apr 12 03:26:00 user nova-compute[72081]:   </os>
Apr 12 03:26:00 user nova-compute[72081]:   <cpu>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='host-passthrough' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='hostPassthroughMigratable'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>off</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='maximum' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='maximumMigratable'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>on</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>off</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='host-model' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <vendor>Intel</vendor>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ss'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='vmx'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='pcid'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='hypervisor'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='arat'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='tsc_adjust'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='umip'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='md-clear'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='stibp'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='arch-capabilities'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ssbd'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ibpb'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ibrs'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='amd-stibp'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='amd-ssbd'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='disable' name='erms'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='disable' name='xsaveopt'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='disable' name='lm'/>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='custom' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>qemu64</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>qemu32</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>phenom</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>pentium3</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>pentium2</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>pentium</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>n270</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>kvm64</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>kvm32</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>coreduo</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>core2duo</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>athlon</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Westmere-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Westmere</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Snowridge</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Server-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Server</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Client-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Client</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>SandyBridge-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>SandyBridge</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Penryn</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G5</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G4</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G3</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G2</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G1</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Nehalem-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Nehalem</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>IvyBridge-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>IvyBridge</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Icelake-Server-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Icelake-Server</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC-Rome</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC-Milan</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC-IBPB</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Dhyana</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Cooperlake</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Conroe</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Cascadelake-Server</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>486</model>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:   </cpu>
Apr 12 03:26:00 user nova-compute[72081]:   <memoryBacking supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:     <enum name='sourceType'>
Apr 12 03:26:00 user nova-compute[72081]:       <value>file</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>anonymous</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>memfd</value>
Apr 12 03:26:00 user nova-compute[72081]:     </enum>
Apr 12 03:26:00 user nova-compute[72081]:   </memoryBacking>
Apr 12 03:26:00 user nova-compute[72081]:   <devices>
Apr 12 03:26:00 user nova-compute[72081]:     <disk supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='diskDevice'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>disk</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>cdrom</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>floppy</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>lun</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='bus'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>fdc</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>scsi</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>usb</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>sata</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='model'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-non-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </disk>
Apr 12 03:26:00 user nova-compute[72081]:     <graphics supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='type'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>sdl</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>vnc</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>spice</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>egl-headless</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </graphics>
Apr 12 03:26:00 user nova-compute[72081]:     <video supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='modelType'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>vga</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>cirrus</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>vmvga</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>qxl</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>none</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>bochs</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>ramfb</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </video>
Apr 12 03:26:00 user nova-compute[72081]:     <hostdev supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='mode'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>subsystem</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='startupPolicy'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>default</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>mandatory</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>requisite</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>optional</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='subsysType'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>usb</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>pci</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>scsi</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='capsType'/>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='pciBackend'/>
Apr 12 03:26:00 user nova-compute[72081]:     </hostdev>
Apr 12 03:26:00 user nova-compute[72081]:     <rng supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='model'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-non-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='backendModel'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>random</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>egd</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>builtin</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </rng>
Apr 12 03:26:00 user nova-compute[72081]:     <filesystem supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='driverType'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>path</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>handle</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtiofs</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </filesystem>
Apr 12 03:26:00 user nova-compute[72081]:     <tpm supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='model'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>tpm-tis</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>tpm-crb</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='backendModel'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>passthrough</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>emulator</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </tpm>
Apr 12 03:26:00 user nova-compute[72081]:   </devices>
Apr 12 03:26:00 user nova-compute[72081]:   <features>
Apr 12 03:26:00 user nova-compute[72081]:     <gic supported='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     <vmcoreinfo supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <genid supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <backingStoreInput supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <backup supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <sev supported='no'/>
Apr 12 03:26:00 user nova-compute[72081]:   </features>
Apr 12 03:26:00 user nova-compute[72081]: </domainCapabilities>
Apr 12 03:26:00 user nova-compute[72081]:  {{(pid=72081) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Libvirt host hypervisor capabilities for arch=i686 and machine_type=ubuntu:
Apr 12 03:26:00 user nova-compute[72081]: <domainCapabilities>
Apr 12 03:26:00 user nova-compute[72081]:   <path>/usr/bin/qemu-system-i386</path>
Apr 12 03:26:00 user nova-compute[72081]:   <domain>kvm</domain>
Apr 12 03:26:00 user nova-compute[72081]:   <machine>pc-i440fx-jammy</machine>
Apr 12 03:26:00 user nova-compute[72081]:   <arch>i686</arch>
Apr 12 03:26:00 user nova-compute[72081]:   <vcpu max='255'/>
Apr 12 03:26:00 user nova-compute[72081]:   <iothreads supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:   <os supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:     <enum name='firmware'/>
Apr 12 03:26:00 user nova-compute[72081]:     <loader supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/OVMF/OVMF_CODE.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/OVMF/OVMF_CODE.secboot.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/AAVMF/AAVMF_CODE.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/AAVMF/AAVMF32_CODE.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/OVMF/OVMF_CODE.ms.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='type'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>rom</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>pflash</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='readonly'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>yes</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>no</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='secure'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>no</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </loader>
Apr 12 03:26:00 user nova-compute[72081]:   </os>
Apr 12 03:26:00 user nova-compute[72081]:   <cpu>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='host-passthrough' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='hostPassthroughMigratable'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>off</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='maximum' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='maximumMigratable'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>on</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>off</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='host-model' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <vendor>Intel</vendor>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ss'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='vmx'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='pcid'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='hypervisor'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='arat'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='tsc_adjust'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='umip'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='md-clear'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='stibp'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='arch-capabilities'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ssbd'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ibpb'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ibrs'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='amd-stibp'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='amd-ssbd'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='disable' name='erms'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='disable' name='xsaveopt'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='disable' name='lm'/>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='custom' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>qemu64</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>qemu32</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>phenom</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>pentium3</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>pentium2</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>pentium</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>n270</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>kvm64</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>kvm32</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>coreduo</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>core2duo</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>athlon</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Westmere-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Westmere</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Snowridge</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Server-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Server</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Client-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Client</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>SandyBridge-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>SandyBridge</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Penryn</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G5</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G4</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G3</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G2</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G1</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Nehalem-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Nehalem</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>IvyBridge-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>IvyBridge</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Icelake-Server-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Icelake-Server</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC-Rome</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC-Milan</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC-IBPB</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Dhyana</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Cooperlake</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Conroe</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Cascadelake-Server</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>486</model>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:   </cpu>
Apr 12 03:26:00 user nova-compute[72081]:   <memoryBacking supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:     <enum name='sourceType'>
Apr 12 03:26:00 user nova-compute[72081]:       <value>file</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>anonymous</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>memfd</value>
Apr 12 03:26:00 user nova-compute[72081]:     </enum>
Apr 12 03:26:00 user nova-compute[72081]:   </memoryBacking>
Apr 12 03:26:00 user nova-compute[72081]:   <devices>
Apr 12 03:26:00 user nova-compute[72081]:     <disk supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='diskDevice'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>disk</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>cdrom</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>floppy</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>lun</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='bus'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>ide</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>fdc</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>scsi</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>usb</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>sata</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='model'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-non-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </disk>
Apr 12 03:26:00 user nova-compute[72081]:     <graphics supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='type'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>sdl</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>vnc</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>spice</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>egl-headless</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </graphics>
Apr 12 03:26:00 user nova-compute[72081]:     <video supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='modelType'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>vga</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>cirrus</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>vmvga</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>qxl</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>none</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>bochs</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>ramfb</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </video>
Apr 12 03:26:00 user nova-compute[72081]:     <hostdev supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='mode'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>subsystem</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='startupPolicy'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>default</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>mandatory</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>requisite</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>optional</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='subsysType'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>usb</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>pci</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>scsi</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='capsType'/>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='pciBackend'/>
Apr 12 03:26:00 user nova-compute[72081]:     </hostdev>
Apr 12 03:26:00 user nova-compute[72081]:     <rng supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='model'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-non-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='backendModel'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>random</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>egd</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>builtin</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </rng>
Apr 12 03:26:00 user nova-compute[72081]:     <filesystem supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='driverType'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>path</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>handle</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtiofs</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </filesystem>
Apr 12 03:26:00 user nova-compute[72081]:     <tpm supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='model'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>tpm-tis</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>tpm-crb</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='backendModel'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>passthrough</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>emulator</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </tpm>
Apr 12 03:26:00 user nova-compute[72081]:   </devices>
Apr 12 03:26:00 user nova-compute[72081]:   <features>
Apr 12 03:26:00 user nova-compute[72081]:     <gic supported='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     <vmcoreinfo supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <genid supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <backingStoreInput supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <backup supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <sev supported='no'/>
Apr 12 03:26:00 user nova-compute[72081]:   </features>
Apr 12 03:26:00 user nova-compute[72081]: </domainCapabilities>
Apr 12 03:26:00 user nova-compute[72081]:  {{(pid=72081) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Libvirt host hypervisor capabilities for arch=i686 and machine_type=q35:
Apr 12 03:26:00 user nova-compute[72081]: <domainCapabilities>
Apr 12 03:26:00 user nova-compute[72081]:   <path>/usr/bin/qemu-system-i386</path>
Apr 12 03:26:00 user nova-compute[72081]:   <domain>kvm</domain>
Apr 12 03:26:00 user nova-compute[72081]:   <machine>pc-q35-6.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:   <arch>i686</arch>
Apr 12 03:26:00 user nova-compute[72081]:   <vcpu max='288'/>
Apr 12 03:26:00 user nova-compute[72081]:   <iothreads supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:   <os supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:     <enum name='firmware'/>
Apr 12 03:26:00 user nova-compute[72081]:     <loader supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/OVMF/OVMF_CODE.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/OVMF/OVMF_CODE.secboot.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/AAVMF/AAVMF_CODE.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/AAVMF/AAVMF32_CODE.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/OVMF/OVMF_CODE.ms.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='type'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>rom</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>pflash</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='readonly'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>yes</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>no</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='secure'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>no</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </loader>
Apr 12 03:26:00 user nova-compute[72081]:   </os>
Apr 12 03:26:00 user nova-compute[72081]:   <cpu>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='host-passthrough' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='hostPassthroughMigratable'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>off</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='maximum' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='maximumMigratable'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>on</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>off</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='host-model' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <vendor>Intel</vendor>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ss'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='vmx'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='pcid'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='hypervisor'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='arat'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='tsc_adjust'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='umip'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='md-clear'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='stibp'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='arch-capabilities'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ssbd'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ibpb'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ibrs'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='amd-stibp'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='amd-ssbd'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='disable' name='erms'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='disable' name='xsaveopt'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='disable' name='lm'/>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='custom' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>qemu64</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>qemu32</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>phenom</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>pentium3</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>pentium2</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>pentium</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>n270</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>kvm64</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>kvm32</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>coreduo</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>core2duo</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>athlon</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Westmere-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Westmere</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Snowridge</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Server-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Server</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Client-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Client</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>SandyBridge-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>SandyBridge</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Penryn</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G5</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G4</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G3</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G2</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G1</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Nehalem-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Nehalem</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>IvyBridge-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>IvyBridge</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Icelake-Server-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Icelake-Server</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC-Rome</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC-Milan</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC-IBPB</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Dhyana</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Cooperlake</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Conroe</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Cascadelake-Server</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>486</model>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:   </cpu>
Apr 12 03:26:00 user nova-compute[72081]:   <memoryBacking supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:     <enum name='sourceType'>
Apr 12 03:26:00 user nova-compute[72081]:       <value>file</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>anonymous</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>memfd</value>
Apr 12 03:26:00 user nova-compute[72081]:     </enum>
Apr 12 03:26:00 user nova-compute[72081]:   </memoryBacking>
Apr 12 03:26:00 user nova-compute[72081]:   <devices>
Apr 12 03:26:00 user nova-compute[72081]:     <disk supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='diskDevice'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>disk</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>cdrom</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>floppy</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>lun</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='bus'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>fdc</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>scsi</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>usb</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>sata</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='model'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-non-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </disk>
Apr 12 03:26:00 user nova-compute[72081]:     <graphics supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='type'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>sdl</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>vnc</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>spice</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>egl-headless</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </graphics>
Apr 12 03:26:00 user nova-compute[72081]:     <video supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='modelType'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>vga</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>cirrus</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>vmvga</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>qxl</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>none</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>bochs</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>ramfb</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </video>
Apr 12 03:26:00 user nova-compute[72081]:     <hostdev supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='mode'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>subsystem</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='startupPolicy'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>default</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>mandatory</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>requisite</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>optional</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='subsysType'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>usb</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>pci</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>scsi</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='capsType'/>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='pciBackend'/>
Apr 12 03:26:00 user nova-compute[72081]:     </hostdev>
Apr 12 03:26:00 user nova-compute[72081]:     <rng supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='model'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-non-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='backendModel'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>random</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>egd</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>builtin</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </rng>
Apr 12 03:26:00 user nova-compute[72081]:     <filesystem supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='driverType'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>path</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>handle</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtiofs</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </filesystem>
Apr 12 03:26:00 user nova-compute[72081]:     <tpm supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='model'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>tpm-tis</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>tpm-crb</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='backendModel'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>passthrough</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>emulator</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </tpm>
Apr 12 03:26:00 user nova-compute[72081]:   </devices>
Apr 12 03:26:00 user nova-compute[72081]:   <features>
Apr 12 03:26:00 user nova-compute[72081]:     <gic supported='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     <vmcoreinfo supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <genid supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <backingStoreInput supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <backup supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <sev supported='no'/>
Apr 12 03:26:00 user nova-compute[72081]:   </features>
Apr 12 03:26:00 user nova-compute[72081]: </domainCapabilities>
Apr 12 03:26:00 user nova-compute[72081]:  {{(pid=72081) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for m68k via machine types: {'virt', None} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch m68k / virt_type kvm / machine_type virt: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-m68k' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch m68k / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-m68k' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for microblaze via machine types: {None} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch microblaze / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-microblaze' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for microblazeel via machine types: {None} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch microblazeel / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-microblazeel' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for mips via machine types: {None} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch mips / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-mips' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for mipsel via machine types: {None} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch mipsel / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-mipsel' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for mips64 via machine types: {None} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch mips64 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-mips64' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for mips64el via machine types: {None} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch mips64el / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-mips64el' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for ppc via machine types: {None} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch ppc / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for ppc64 via machine types: {'powernv', 'pseries', None} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch ppc64 / virt_type kvm / machine_type powernv: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch ppc64 / virt_type kvm / machine_type pseries: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch ppc64 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for ppc64le via machine types: {'powernv', 'pseries'} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch ppc64le / virt_type kvm / machine_type powernv: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64le' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch ppc64le / virt_type kvm / machine_type pseries: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64le' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for riscv32 via machine types: {None} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch riscv32 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-riscv32' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for riscv64 via machine types: {None} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch riscv64 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-riscv64' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for s390x via machine types: {'s390-ccw-virtio'} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch s390x / virt_type kvm / machine_type s390-ccw-virtio: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-s390x' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for sh4 via machine types: {None} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch sh4 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-sh4' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for sh4eb via machine types: {None} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch sh4eb / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-sh4eb' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for sparc via machine types: {None} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch sparc / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-sparc' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for sparc64 via machine types: {None} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch sparc64 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-sparc64' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for x86_64 via machine types: {'pc', 'ubuntu-q35', 'ubuntu', 'q35'} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Libvirt host hypervisor capabilities for arch=x86_64 and machine_type=pc:
Apr 12 03:26:00 user nova-compute[72081]: <domainCapabilities>
Apr 12 03:26:00 user nova-compute[72081]:   <path>/usr/bin/qemu-system-x86_64</path>
Apr 12 03:26:00 user nova-compute[72081]:   <domain>kvm</domain>
Apr 12 03:26:00 user nova-compute[72081]:   <machine>pc-i440fx-6.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:   <arch>x86_64</arch>
Apr 12 03:26:00 user nova-compute[72081]:   <vcpu max='255'/>
Apr 12 03:26:00 user nova-compute[72081]:   <iothreads supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:   <os supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:     <enum name='firmware'>
Apr 12 03:26:00 user nova-compute[72081]:       <value>efi</value>
Apr 12 03:26:00 user nova-compute[72081]:     </enum>
Apr 12 03:26:00 user nova-compute[72081]:     <loader supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/OVMF/OVMF_CODE_4M.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='type'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>rom</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>pflash</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='readonly'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>yes</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>no</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='secure'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>no</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </loader>
Apr 12 03:26:00 user nova-compute[72081]:   </os>
Apr 12 03:26:00 user nova-compute[72081]:   <cpu>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='host-passthrough' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='hostPassthroughMigratable'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>on</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>off</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='maximum' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='maximumMigratable'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>on</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>off</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='host-model' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <vendor>Intel</vendor>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ss'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='vmx'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='pcid'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='hypervisor'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='arat'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='tsc_adjust'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='umip'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='md-clear'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='stibp'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='arch-capabilities'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ssbd'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='invtsc'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ibpb'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ibrs'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='amd-stibp'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='amd-ssbd'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='rsba'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='skip-l1dfl-vmentry'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='pschange-mc-no'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='disable' name='erms'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='disable' name='xsaveopt'/>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='custom' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>qemu64</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>qemu32</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>phenom</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>pentium3</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>pentium2</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>pentium</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>n270</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>kvm64</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>kvm32</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>coreduo</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>core2duo</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>athlon</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Westmere-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Westmere</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Snowridge</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Server-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Server</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Client-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Client</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>SandyBridge-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>SandyBridge</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Penryn</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G5</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G4</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G3</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G2</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Opteron_G1</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Nehalem-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Nehalem</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>IvyBridge-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>IvyBridge</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Icelake-Server-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Icelake-Server</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC-Rome</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC-Milan</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC-IBPB</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Dhyana</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Cooperlake</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Conroe</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Cascadelake-Server</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>486</model>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:   </cpu>
Apr 12 03:26:00 user nova-compute[72081]:   <memoryBacking supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:     <enum name='sourceType'>
Apr 12 03:26:00 user nova-compute[72081]:       <value>file</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>anonymous</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>memfd</value>
Apr 12 03:26:00 user nova-compute[72081]:     </enum>
Apr 12 03:26:00 user nova-compute[72081]:   </memoryBacking>
Apr 12 03:26:00 user nova-compute[72081]:   <devices>
Apr 12 03:26:00 user nova-compute[72081]:     <disk supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='diskDevice'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>disk</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>cdrom</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>floppy</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>lun</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='bus'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>ide</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>fdc</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>scsi</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>usb</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>sata</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='model'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-non-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </disk>
Apr 12 03:26:00 user nova-compute[72081]:     <graphics supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='type'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>sdl</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>vnc</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>spice</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>egl-headless</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </graphics>
Apr 12 03:26:00 user nova-compute[72081]:     <video supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='modelType'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>vga</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>cirrus</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>vmvga</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>qxl</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>none</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>bochs</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>ramfb</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </video>
Apr 12 03:26:00 user nova-compute[72081]:     <hostdev supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='mode'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>subsystem</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='startupPolicy'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>default</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>mandatory</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>requisite</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>optional</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='subsysType'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>usb</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>pci</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>scsi</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='capsType'/>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='pciBackend'/>
Apr 12 03:26:00 user nova-compute[72081]:     </hostdev>
Apr 12 03:26:00 user nova-compute[72081]:     <rng supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='model'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-non-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='backendModel'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>random</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>egd</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>builtin</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </rng>
Apr 12 03:26:00 user nova-compute[72081]:     <filesystem supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='driverType'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>path</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>handle</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtiofs</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </filesystem>
Apr 12 03:26:00 user nova-compute[72081]:     <tpm supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='model'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>tpm-tis</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>tpm-crb</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='backendModel'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>passthrough</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>emulator</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </tpm>
Apr 12 03:26:00 user nova-compute[72081]:   </devices>
Apr 12 03:26:00 user nova-compute[72081]:   <features>
Apr 12 03:26:00 user nova-compute[72081]:     <gic supported='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     <vmcoreinfo supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <genid supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <backingStoreInput supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <backup supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <sev supported='no'/>
Apr 12 03:26:00 user nova-compute[72081]:   </features>
Apr 12 03:26:00 user nova-compute[72081]: </domainCapabilities>
Apr 12 03:26:00 user nova-compute[72081]:  {{(pid=72081) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Libvirt host hypervisor capabilities for arch=x86_64 and machine_type=ubuntu-q35:
Apr 12 03:26:00 user nova-compute[72081]: <domainCapabilities>
Apr 12 03:26:00 user nova-compute[72081]:   <path>/usr/bin/qemu-system-x86_64</path>
Apr 12 03:26:00 user nova-compute[72081]:   <domain>kvm</domain>
Apr 12 03:26:00 user nova-compute[72081]:   <machine>pc-q35-jammy</machine>
Apr 12 03:26:00 user nova-compute[72081]:   <arch>x86_64</arch>
Apr 12 03:26:00 user nova-compute[72081]:   <vcpu max='288'/>
Apr 12 03:26:00 user nova-compute[72081]:   <iothreads supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:   <os supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:     <enum name='firmware'>
Apr 12 03:26:00 user nova-compute[72081]:       <value>efi</value>
Apr 12 03:26:00 user nova-compute[72081]:     </enum>
Apr 12 03:26:00 user nova-compute[72081]:     <loader supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/OVMF/OVMF_CODE_4M.ms.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/OVMF/OVMF_CODE_4M.secboot.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/OVMF/OVMF_CODE_4M.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='type'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>rom</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>pflash</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='readonly'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>yes</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>no</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='secure'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>yes</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>no</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </loader>
Apr 12 03:26:00 user nova-compute[72081]:   </os>
Apr 12 03:26:00 user nova-compute[72081]:   <cpu>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='host-passthrough' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='hostPassthroughMigratable'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>on</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>off</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='maximum' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='maximumMigratable'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>on</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>off</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='host-model' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <vendor>Intel</vendor>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ss'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='vmx'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='pcid'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='hypervisor'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='arat'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='tsc_adjust'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='umip'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='md-clear'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='stibp'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='arch-capabilities'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ssbd'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='invtsc'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ibpb'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ibrs'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='amd-stibp'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='amd-ssbd'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='rsba'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='skip-l1dfl-vmentry'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='pschange-mc-no'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='disable' name='erms'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='disable' name='xsaveopt'/>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='custom' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>qemu64</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>qemu32</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>phenom</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>pentium3</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>pentium2</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>pentium</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>n270</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>kvm64</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>kvm32</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>coreduo</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>core2duo</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>athlon</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Westmere-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Westmere</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Snowridge</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Server-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Server</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Client-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Client</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>SandyBridge-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>SandyBridge</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Penryn</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G5</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G4</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G3</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G2</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Opteron_G1</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Nehalem-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Nehalem</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>IvyBridge-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>IvyBridge</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Icelake-Server-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Icelake-Server</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC-Rome</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC-Milan</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC-IBPB</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Dhyana</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Cooperlake</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Conroe</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Cascadelake-Server</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>486</model>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:   </cpu>
Apr 12 03:26:00 user nova-compute[72081]:   <memoryBacking supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:     <enum name='sourceType'>
Apr 12 03:26:00 user nova-compute[72081]:       <value>file</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>anonymous</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>memfd</value>
Apr 12 03:26:00 user nova-compute[72081]:     </enum>
Apr 12 03:26:00 user nova-compute[72081]:   </memoryBacking>
Apr 12 03:26:00 user nova-compute[72081]:   <devices>
Apr 12 03:26:00 user nova-compute[72081]:     <disk supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='diskDevice'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>disk</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>cdrom</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>floppy</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>lun</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='bus'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>fdc</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>scsi</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>usb</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>sata</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='model'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-non-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </disk>
Apr 12 03:26:00 user nova-compute[72081]:     <graphics supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='type'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>sdl</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>vnc</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>spice</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>egl-headless</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </graphics>
Apr 12 03:26:00 user nova-compute[72081]:     <video supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='modelType'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>vga</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>cirrus</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>vmvga</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>qxl</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>none</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>bochs</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>ramfb</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </video>
Apr 12 03:26:00 user nova-compute[72081]:     <hostdev supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='mode'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>subsystem</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='startupPolicy'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>default</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>mandatory</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>requisite</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>optional</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='subsysType'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>usb</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>pci</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>scsi</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='capsType'/>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='pciBackend'/>
Apr 12 03:26:00 user nova-compute[72081]:     </hostdev>
Apr 12 03:26:00 user nova-compute[72081]:     <rng supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='model'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-non-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='backendModel'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>random</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>egd</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>builtin</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </rng>
Apr 12 03:26:00 user nova-compute[72081]:     <filesystem supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='driverType'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>path</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>handle</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtiofs</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </filesystem>
Apr 12 03:26:00 user nova-compute[72081]:     <tpm supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='model'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>tpm-tis</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>tpm-crb</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='backendModel'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>passthrough</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>emulator</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </tpm>
Apr 12 03:26:00 user nova-compute[72081]:   </devices>
Apr 12 03:26:00 user nova-compute[72081]:   <features>
Apr 12 03:26:00 user nova-compute[72081]:     <gic supported='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     <vmcoreinfo supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <genid supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <backingStoreInput supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <backup supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <sev supported='no'/>
Apr 12 03:26:00 user nova-compute[72081]:   </features>
Apr 12 03:26:00 user nova-compute[72081]: </domainCapabilities>
Apr 12 03:26:00 user nova-compute[72081]:  {{(pid=72081) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Libvirt host hypervisor capabilities for arch=x86_64 and machine_type=ubuntu:
Apr 12 03:26:00 user nova-compute[72081]: <domainCapabilities>
Apr 12 03:26:00 user nova-compute[72081]:   <path>/usr/bin/qemu-system-x86_64</path>
Apr 12 03:26:00 user nova-compute[72081]:   <domain>kvm</domain>
Apr 12 03:26:00 user nova-compute[72081]:   <machine>pc-i440fx-jammy</machine>
Apr 12 03:26:00 user nova-compute[72081]:   <arch>x86_64</arch>
Apr 12 03:26:00 user nova-compute[72081]:   <vcpu max='255'/>
Apr 12 03:26:00 user nova-compute[72081]:   <iothreads supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:   <os supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:     <enum name='firmware'>
Apr 12 03:26:00 user nova-compute[72081]:       <value>efi</value>
Apr 12 03:26:00 user nova-compute[72081]:     </enum>
Apr 12 03:26:00 user nova-compute[72081]:     <loader supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/OVMF/OVMF_CODE_4M.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='type'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>rom</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>pflash</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='readonly'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>yes</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>no</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='secure'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>no</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </loader>
Apr 12 03:26:00 user nova-compute[72081]:   </os>
Apr 12 03:26:00 user nova-compute[72081]:   <cpu>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='host-passthrough' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='hostPassthroughMigratable'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>on</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>off</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='maximum' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='maximumMigratable'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>on</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>off</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='host-model' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <vendor>Intel</vendor>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ss'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='vmx'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='pcid'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='hypervisor'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='arat'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='tsc_adjust'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='umip'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='md-clear'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='stibp'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='arch-capabilities'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ssbd'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='invtsc'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ibpb'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ibrs'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='amd-stibp'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='amd-ssbd'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='rsba'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='skip-l1dfl-vmentry'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='pschange-mc-no'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='disable' name='erms'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='disable' name='xsaveopt'/>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='custom' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>qemu64</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>qemu32</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>phenom</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>pentium3</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>pentium2</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>pentium</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>n270</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>kvm64</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>kvm32</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>coreduo</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>core2duo</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>athlon</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Westmere-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Westmere</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Snowridge</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Server-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Server</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Client-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Client</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>SandyBridge-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>SandyBridge</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Penryn</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G5</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G4</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G3</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G2</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Opteron_G1</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Nehalem-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Nehalem</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>IvyBridge-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>IvyBridge</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Icelake-Server-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Icelake-Server</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC-Rome</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC-Milan</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC-IBPB</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Dhyana</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Cooperlake</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Conroe</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Cascadelake-Server</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>486</model>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:   </cpu>
Apr 12 03:26:00 user nova-compute[72081]:   <memoryBacking supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:     <enum name='sourceType'>
Apr 12 03:26:00 user nova-compute[72081]:       <value>file</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>anonymous</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>memfd</value>
Apr 12 03:26:00 user nova-compute[72081]:     </enum>
Apr 12 03:26:00 user nova-compute[72081]:   </memoryBacking>
Apr 12 03:26:00 user nova-compute[72081]:   <devices>
Apr 12 03:26:00 user nova-compute[72081]:     <disk supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='diskDevice'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>disk</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>cdrom</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>floppy</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>lun</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='bus'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>ide</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>fdc</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>scsi</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>usb</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>sata</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='model'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-non-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </disk>
Apr 12 03:26:00 user nova-compute[72081]:     <graphics supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='type'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>sdl</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>vnc</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>spice</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>egl-headless</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </graphics>
Apr 12 03:26:00 user nova-compute[72081]:     <video supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='modelType'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>vga</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>cirrus</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>vmvga</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>qxl</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>none</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>bochs</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>ramfb</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </video>
Apr 12 03:26:00 user nova-compute[72081]:     <hostdev supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='mode'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>subsystem</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='startupPolicy'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>default</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>mandatory</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>requisite</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>optional</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='subsysType'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>usb</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>pci</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>scsi</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='capsType'/>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='pciBackend'/>
Apr 12 03:26:00 user nova-compute[72081]:     </hostdev>
Apr 12 03:26:00 user nova-compute[72081]:     <rng supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='model'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-non-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='backendModel'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>random</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>egd</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>builtin</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </rng>
Apr 12 03:26:00 user nova-compute[72081]:     <filesystem supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='driverType'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>path</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>handle</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtiofs</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </filesystem>
Apr 12 03:26:00 user nova-compute[72081]:     <tpm supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='model'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>tpm-tis</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>tpm-crb</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='backendModel'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>passthrough</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>emulator</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </tpm>
Apr 12 03:26:00 user nova-compute[72081]:   </devices>
Apr 12 03:26:00 user nova-compute[72081]:   <features>
Apr 12 03:26:00 user nova-compute[72081]:     <gic supported='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     <vmcoreinfo supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <genid supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <backingStoreInput supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <backup supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <sev supported='no'/>
Apr 12 03:26:00 user nova-compute[72081]:   </features>
Apr 12 03:26:00 user nova-compute[72081]: </domainCapabilities>
Apr 12 03:26:00 user nova-compute[72081]:  {{(pid=72081) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Libvirt host hypervisor capabilities for arch=x86_64 and machine_type=q35:
Apr 12 03:26:00 user nova-compute[72081]: <domainCapabilities>
Apr 12 03:26:00 user nova-compute[72081]:   <path>/usr/bin/qemu-system-x86_64</path>
Apr 12 03:26:00 user nova-compute[72081]:   <domain>kvm</domain>
Apr 12 03:26:00 user nova-compute[72081]:   <machine>pc-q35-6.2</machine>
Apr 12 03:26:00 user nova-compute[72081]:   <arch>x86_64</arch>
Apr 12 03:26:00 user nova-compute[72081]:   <vcpu max='288'/>
Apr 12 03:26:00 user nova-compute[72081]:   <iothreads supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:   <os supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:     <enum name='firmware'>
Apr 12 03:26:00 user nova-compute[72081]:       <value>efi</value>
Apr 12 03:26:00 user nova-compute[72081]:     </enum>
Apr 12 03:26:00 user nova-compute[72081]:     <loader supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/OVMF/OVMF_CODE_4M.ms.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/OVMF/OVMF_CODE_4M.secboot.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>/usr/share/OVMF/OVMF_CODE_4M.fd</value>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='type'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>rom</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>pflash</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='readonly'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>yes</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>no</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='secure'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>yes</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>no</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </loader>
Apr 12 03:26:00 user nova-compute[72081]:   </os>
Apr 12 03:26:00 user nova-compute[72081]:   <cpu>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='host-passthrough' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='hostPassthroughMigratable'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>on</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>off</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='maximum' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='maximumMigratable'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>on</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>off</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='host-model' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <vendor>Intel</vendor>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ss'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='vmx'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='pcid'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='hypervisor'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='arat'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='tsc_adjust'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='umip'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='md-clear'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='stibp'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='arch-capabilities'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ssbd'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='invtsc'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ibpb'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='ibrs'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='amd-stibp'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='amd-ssbd'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='rsba'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='skip-l1dfl-vmentry'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='require' name='pschange-mc-no'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='disable' name='erms'/>
Apr 12 03:26:00 user nova-compute[72081]:       <feature policy='disable' name='xsaveopt'/>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:     <mode name='custom' supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>qemu64</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>qemu32</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>phenom</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>pentium3</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>pentium2</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>pentium</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>n270</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>kvm64</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>kvm32</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>coreduo</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>core2duo</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>athlon</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Westmere-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Westmere</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Snowridge</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Server-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Server</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Client-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Skylake-Client</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>SandyBridge-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>SandyBridge</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Penryn</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G5</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G4</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G3</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Opteron_G2</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Opteron_G1</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Nehalem-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Nehalem</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>IvyBridge-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>IvyBridge</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Icelake-Server-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Icelake-Server</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Haswell</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC-Rome</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC-Milan</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC-IBPB</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>EPYC</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Dhyana</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Cooperlake</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>Conroe</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Cascadelake-Server</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell-noTSX</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell-IBRS</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='no'>Broadwell</model>
Apr 12 03:26:00 user nova-compute[72081]:       <model usable='yes'>486</model>
Apr 12 03:26:00 user nova-compute[72081]:     </mode>
Apr 12 03:26:00 user nova-compute[72081]:   </cpu>
Apr 12 03:26:00 user nova-compute[72081]:   <memoryBacking supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:     <enum name='sourceType'>
Apr 12 03:26:00 user nova-compute[72081]:       <value>file</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>anonymous</value>
Apr 12 03:26:00 user nova-compute[72081]:       <value>memfd</value>
Apr 12 03:26:00 user nova-compute[72081]:     </enum>
Apr 12 03:26:00 user nova-compute[72081]:   </memoryBacking>
Apr 12 03:26:00 user nova-compute[72081]:   <devices>
Apr 12 03:26:00 user nova-compute[72081]:     <disk supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='diskDevice'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>disk</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>cdrom</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>floppy</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>lun</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='bus'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>fdc</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>scsi</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>usb</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>sata</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='model'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-non-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </disk>
Apr 12 03:26:00 user nova-compute[72081]:     <graphics supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='type'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>sdl</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>vnc</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>spice</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>egl-headless</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </graphics>
Apr 12 03:26:00 user nova-compute[72081]:     <video supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='modelType'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>vga</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>cirrus</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>vmvga</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>qxl</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>none</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>bochs</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>ramfb</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </video>
Apr 12 03:26:00 user nova-compute[72081]:     <hostdev supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='mode'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>subsystem</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='startupPolicy'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>default</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>mandatory</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>requisite</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>optional</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='subsysType'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>usb</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>pci</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>scsi</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='capsType'/>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='pciBackend'/>
Apr 12 03:26:00 user nova-compute[72081]:     </hostdev>
Apr 12 03:26:00 user nova-compute[72081]:     <rng supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='model'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtio-non-transitional</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='backendModel'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>random</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>egd</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>builtin</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </rng>
Apr 12 03:26:00 user nova-compute[72081]:     <filesystem supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='driverType'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>path</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>handle</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>virtiofs</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </filesystem>
Apr 12 03:26:00 user nova-compute[72081]:     <tpm supported='yes'>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='model'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>tpm-tis</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>tpm-crb</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:       <enum name='backendModel'>
Apr 12 03:26:00 user nova-compute[72081]:         <value>passthrough</value>
Apr 12 03:26:00 user nova-compute[72081]:         <value>emulator</value>
Apr 12 03:26:00 user nova-compute[72081]:       </enum>
Apr 12 03:26:00 user nova-compute[72081]:     </tpm>
Apr 12 03:26:00 user nova-compute[72081]:   </devices>
Apr 12 03:26:00 user nova-compute[72081]:   <features>
Apr 12 03:26:00 user nova-compute[72081]:     <gic supported='no'/>
Apr 12 03:26:00 user nova-compute[72081]:     <vmcoreinfo supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <genid supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <backingStoreInput supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <backup supported='yes'/>
Apr 12 03:26:00 user nova-compute[72081]:     <sev supported='no'/>
Apr 12 03:26:00 user nova-compute[72081]:   </features>
Apr 12 03:26:00 user nova-compute[72081]: </domainCapabilities>
Apr 12 03:26:00 user nova-compute[72081]:  {{(pid=72081) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for xtensa via machine types: {None} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch xtensa / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-xtensa' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Getting domain capabilities for xtensaeb via machine types: {None} {{(pid=72081) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Error from libvirt when retrieving domain capabilities for arch xtensaeb / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-xtensaeb' on this host {{(pid=72081) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Checking secure boot support for host arch (x86_64) {{(pid=72081) supports_secure_boot /opt/stack/nova/nova/virt/libvirt/host.py:1750}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Checking secure boot support for host arch (x86_64) {{(pid=72081) supports_secure_boot /opt/stack/nova/nova/virt/libvirt/host.py:1750}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Checking secure boot support for host arch (x86_64) {{(pid=72081) supports_secure_boot /opt/stack/nova/nova/virt/libvirt/host.py:1750}}
Apr 12 03:26:00 user nova-compute[72081]: INFO nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Secure Boot support detected
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] cpu compare xml: <cpu match="exact">
Apr 12 03:26:00 user nova-compute[72081]:   <model>Nehalem</model>
Apr 12 03:26:00 user nova-compute[72081]: </cpu>
Apr 12 03:26:00 user nova-compute[72081]:  {{(pid=72081) _compare_cpu /opt/stack/nova/nova/virt/libvirt/driver.py:9996}}
Apr 12 03:26:00 user nova-compute[72081]: INFO nova.virt.node [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Generated node identity 2203ce45-27b6-4606-b33a-0f78fa74c5d6
Apr 12 03:26:00 user nova-compute[72081]: INFO nova.virt.node [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Wrote node identity 2203ce45-27b6-4606-b33a-0f78fa74c5d6 to /opt/stack/data/nova/compute_id
Apr 12 03:26:00 user nova-compute[72081]: WARNING nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Compute nodes ['2203ce45-27b6-4606-b33a-0f78fa74c5d6'] for host user were not found in the database. If this is the first time this service is starting on this host, then you can ignore this warning.
Apr 12 03:26:00 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Looking for unclaimed instances stuck in BUILDING status for nodes managed by this host
Apr 12 03:26:00 user nova-compute[72081]: WARNING nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] No compute node record found for host user. If this is the first time this service is starting on this host, then you can ignore this warning.: nova.exception_Remote.ComputeHostNotFound_Remote: Compute host user could not be found.
Apr 12 03:26:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:26:00 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:26:01 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:26:01 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:26:01 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Hypervisor/Node resource view: name=user free_ram=10841MB free_disk=26.941993713378906GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:26:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:26:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:26:01 user nova-compute[72081]: WARNING nova.compute.resource_tracker [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] No compute node record for user:2203ce45-27b6-4606-b33a-0f78fa74c5d6: nova.exception_Remote.ComputeHostNotFound_Remote: Compute host 2203ce45-27b6-4606-b33a-0f78fa74c5d6 could not be found.
Apr 12 03:26:01 user nova-compute[72081]: INFO nova.compute.resource_tracker [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Compute node record created for user:user with uuid: 2203ce45-27b6-4606-b33a-0f78fa74c5d6
Apr 12 03:26:01 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:26:01 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:26:01 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [req-eed7c6cf-ad07-485e-8ad5-5336fb0f49ea] Created resource provider record via placement API for resource provider with UUID 2203ce45-27b6-4606-b33a-0f78fa74c5d6 and name user.
Apr 12 03:26:01 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] /sys/module/kvm_amd/parameters/sev does not exist {{(pid=72081) _kernel_supports_amd_sev /opt/stack/nova/nova/virt/libvirt/host.py:1766}}
Apr 12 03:26:01 user nova-compute[72081]: INFO nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] kernel doesn't support AMD SEV
Apr 12 03:26:01 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Updating inventory in ProviderTree for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 with inventory: {'MEMORY_MB': {'total': 16023, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0, 'reserved': 512}, 'VCPU': {'total': 12, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0, 'reserved': 0}, 'DISK_GB': {'total': 40, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0, 'reserved': 0}} {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 12 03:26:01 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:26:01 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Libvirt baseline CPU <cpu>
Apr 12 03:26:01 user nova-compute[72081]:   <arch>x86_64</arch>
Apr 12 03:26:01 user nova-compute[72081]:   <model>Nehalem</model>
Apr 12 03:26:01 user nova-compute[72081]:   <vendor>Intel</vendor>
Apr 12 03:26:01 user nova-compute[72081]:   <topology sockets="6" cores="1" threads="1"/>
Apr 12 03:26:01 user nova-compute[72081]: </cpu>
Apr 12 03:26:01 user nova-compute[72081]:  {{(pid=72081) _get_guest_baseline_cpu_features /opt/stack/nova/nova/virt/libvirt/driver.py:12486}}
Apr 12 03:26:01 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Updated inventory for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 with generation 0 in Placement from set_inventory_for_provider using data: {'MEMORY_MB': {'total': 16023, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0, 'reserved': 512}, 'VCPU': {'total': 12, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0, 'reserved': 0}, 'DISK_GB': {'total': 40, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0, 'reserved': 0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:957}}
Apr 12 03:26:01 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Updating resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 generation from 0 to 1 during operation: update_inventory {{(pid=72081) _update_generation /opt/stack/nova/nova/compute/provider_tree.py:164}}
Apr 12 03:26:01 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Updating inventory in ProviderTree for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 with inventory: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 12 03:26:01 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Updating resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 generation from 1 to 2 during operation: update_traits {{(pid=72081) _update_generation /opt/stack/nova/nova/compute/provider_tree.py:164}}
Apr 12 03:26:01 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:26:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.794s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:26:01 user nova-compute[72081]: DEBUG nova.service [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Creating RPC server for service compute {{(pid=72081) start /opt/stack/nova/nova/service.py:182}}
Apr 12 03:26:01 user nova-compute[72081]: DEBUG nova.service [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Join ServiceGroup membership for this service compute {{(pid=72081) start /opt/stack/nova/nova/service.py:199}}
Apr 12 03:26:01 user nova-compute[72081]: DEBUG nova.servicegroup.drivers.db [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] DB_Driver: join new ServiceGroup member user to the compute group, service = <Service: host=user, binary=nova-compute, manager_class_name=nova.compute.manager.ComputeManager> {{(pid=72081) join /opt/stack/nova/nova/servicegroup/drivers/db.py:44}}
Apr 12 03:26:50 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_power_states {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:26:50 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._cleanup_running_deleted_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:26:52 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:26:52 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:26:52 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:26:52 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Rebuilding the list of instances to heal {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 12 03:26:52 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Didn't find any instances for network info cache update. {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 12 03:26:52 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:26:52 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:26:52 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:26:52 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:26:52 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:26:52 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:26:52 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:26:52 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:26:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:26:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:26:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:26:52 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:26:52 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:26:52 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:26:53 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=10248MB free_disk=26.85572052001953GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:26:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:26:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:26:53 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:26:53 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:26:53 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:26:53 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:26:53 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:26:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.106s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:27:53 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:27:53 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:27:53 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:27:53 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:27:53 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:27:53 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:27:53 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:27:53 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:27:53 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Rebuilding the list of instances to heal {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 12 03:27:53 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Didn't find any instances for network info cache update. {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 12 03:27:53 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:27:54 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:27:54 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:27:54 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:27:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:27:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:27:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:27:54 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:27:54 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:27:54 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:27:54 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=10213MB free_disk=26.902908325195312GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:27:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:27:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:27:55 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:27:55 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:27:55 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:27:55 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:27:55 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:27:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.135s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:28:53 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:28:53 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:28:53 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:28:53 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:28:53 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:28:54 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:28:54 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:28:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:28:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:28:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:28:54 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:28:54 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:28:54 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:28:54 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=10174MB free_disk=26.677963256835938GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:28:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:28:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:28:55 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:28:55 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:28:55 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:28:55 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:28:55 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:28:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.101s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:28:56 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:28:56 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:28:56 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Rebuilding the list of instances to heal {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 12 03:28:56 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Didn't find any instances for network info cache update. {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 12 03:28:56 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:28:56 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:29:52 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:29:53 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:29:53 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:29:53 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:29:54 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:29:55 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:29:55 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:29:55 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:29:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:29:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:29:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:29:55 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:29:56 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:29:56 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:29:56 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=9445MB free_disk=26.70053482055664GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:29:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:29:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:29:56 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:29:56 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:29:56 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:29:56 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:29:56 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:29:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.140s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:29:57 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:29:57 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:29:57 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Rebuilding the list of instances to heal {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 12 03:29:57 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Didn't find any instances for network info cache update. {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 12 03:29:57 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:29:57 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:30:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Acquiring lock "56834002-7593-4592-a86d-23bb90d00be0" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "56834002-7593-4592-a86d-23bb90d00be0" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:30:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:30:10 user nova-compute[72081]: INFO nova.compute.claims [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Claim successful on node user
Apr 12 03:30:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "2d0904c3-74e0-47bb-a832-758a7035b2f5" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "2d0904c3-74e0-47bb-a832-758a7035b2f5" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:30:10 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:30:10 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:30:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.459s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:30:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.039s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:30:10 user nova-compute[72081]: INFO nova.compute.claims [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Claim successful on node user
Apr 12 03:30:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:30:10 user nova-compute[72081]: DEBUG nova.network.neutron [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:30:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:11 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:30:11 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:30:11 user nova-compute[72081]: DEBUG nova.compute.manager [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:30:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:11 user nova-compute[72081]: DEBUG nova.compute.manager [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:30:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:30:11 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Creating image(s)
Apr 12 03:30:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Acquiring lock "/opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "/opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "/opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.005s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.003s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:11 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:30:11 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:30:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.597s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:11 user nova-compute[72081]: DEBUG nova.compute.manager [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:30:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.232s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:11 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:30:11 user nova-compute[72081]: INFO nova.compute.claims [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Claim successful on node user
Apr 12 03:30:11 user nova-compute[72081]: DEBUG nova.compute.manager [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:30:11 user nova-compute[72081]: DEBUG nova.network.neutron [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:30:11 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:30:11 user nova-compute[72081]: DEBUG nova.compute.manager [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:30:11 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df.part --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG nova.policy [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '3da7394c51664a7caacfda6b7dda9066', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '83a7ef3f97fd47849973d4e4482fb5f9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG nova.compute.manager [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:30:12 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Creating image(s)
Apr 12 03:30:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "/opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "/opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "/opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG nova.policy [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd99be569057f432484e4a60b46882bbd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e1fa352a8c144a1aada7ce0d8818429', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.773s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df.part --force-share --output=json" returned: 0 in 0.178s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG nova.virt.images [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7 was qcow2, converting to raw {{(pid=72081) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG nova.privsep.utils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72081) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df.part /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df.converted {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:30:12 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:30:12 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df.part /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df.converted" returned: 0 in 0.131s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df.converted --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:30:12 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Creating image(s)
Apr 12 03:30:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "/opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "/opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "/opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df.converted --force-share --output=json" returned: 0 in 0.154s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.221s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:12 user nova-compute[72081]: INFO oslo.privsep.daemon [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Running privsep helper: ['sudo', 'nova-rootwrap', '/etc/nova/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/nova/nova-cpu.conf', '--privsep_context', 'nova.privsep.sys_admin_pctxt', '--privsep_sock_path', '/tmp/tmpn4mpsaxm/privsep.sock']
Apr 12 03:30:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.335s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.034s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:12 user sudo[80815]:    stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/nova-rootwrap /etc/nova/rootwrap.conf privsep-helper --config-file /etc/nova/nova-cpu.conf --privsep_context nova.privsep.sys_admin_pctxt --privsep_sock_path /tmp/tmpn4mpsaxm/privsep.sock
Apr 12 03:30:12 user sudo[80815]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1001)
Apr 12 03:30:12 user nova-compute[72081]: DEBUG nova.policy [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'de1fe5db01624224ab7b615c908d990b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '738ebb6b798c4974aa46f9bbe3828279', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:30:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Acquiring lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:13 user nova-compute[72081]: DEBUG nova.compute.manager [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:30:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:13 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:30:13 user nova-compute[72081]: INFO nova.compute.claims [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Claim successful on node user
Apr 12 03:30:14 user sudo[80815]: pam_unix(sudo:session): session closed for user root
Apr 12 03:30:14 user nova-compute[72081]: INFO oslo.privsep.daemon [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Spawned new privsep daemon via rootwrap
Apr 12 03:30:14 user nova-compute[72081]: INFO oslo.privsep.daemon [-] privsep daemon starting
Apr 12 03:30:14 user nova-compute[72081]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
Apr 12 03:30:14 user nova-compute[72081]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_CHOWN|CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_FOWNER|CAP_NET_ADMIN|CAP_SYS_ADMIN/CAP_CHOWN|CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_FOWNER|CAP_NET_ADMIN|CAP_SYS_ADMIN/none
Apr 12 03:30:14 user nova-compute[72081]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 80819
Apr 12 03:30:14 user nova-compute[72081]: WARNING oslo_privsep.priv_context [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] privsep daemon already running
Apr 12 03:30:14 user nova-compute[72081]: WARNING oslo_privsep.priv_context [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] privsep daemon already running
Apr 12 03:30:14 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.501s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG nova.network.neutron [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:30:14 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:30:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.154s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG nova.network.neutron [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Successfully created port: dce84390-a2f8-4047-b513-eec5845a2399 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.165s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.158s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.130s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk 1073741824" returned: 0 in 0.052s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.186s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.172s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:30:14 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Creating image(s)
Apr 12 03:30:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Acquiring lock "/opt/stack/data/nova/instances/ef4ab4cb-6164-47ea-be1d-cbe0698f991d/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "/opt/stack/data/nova/instances/ef4ab4cb-6164-47ea-be1d-cbe0698f991d/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "/opt/stack/data/nova/instances/ef4ab4cb-6164-47ea-be1d-cbe0698f991d/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.034s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG nova.network.neutron [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Successfully created port: 8f7b221a-25ae-41f9-847d-5aedf2d93a71 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:30:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.158s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Checking if we can resize image /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.175s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.130s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk 1073741824" returned: 0 in 0.048s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.227s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.401s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json" returned: 0 in 0.163s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Cannot resize image /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG nova.objects.instance [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lazy-loading 'migration_context' on Instance uuid 56834002-7593-4592-a86d-23bb90d00be0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Ensure instance console log exists: /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG nova.policy [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '4d01e7aaa3734b299d0c0a27844011b4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5dadbf1b0a124d2d929c85bc2b916022', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.146s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Checking if we can resize image /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.156s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk 1073741824" returned: 0 in 0.043s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.206s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.272s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Cannot resize image /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG nova.objects.instance [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lazy-loading 'migration_context' on Instance uuid 2d0904c3-74e0-47bb-a832-758a7035b2f5 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Ensure instance console log exists: /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Checking if we can resize image /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.146s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/ef4ab4cb-6164-47ea-be1d-cbe0698f991d/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/ef4ab4cb-6164-47ea-be1d-cbe0698f991d/disk 1073741824" returned: 0 in 0.058s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Acquiring lock "72ba1218-065d-40c4-8503-7c4155e47503" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "72ba1218-065d-40c4-8503-7c4155e47503" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.213s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG nova.compute.manager [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Cannot resize image /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG nova.objects.instance [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lazy-loading 'migration_context' on Instance uuid b23d4bcd-989a-45e1-89a3-6e3d30229a32 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Ensure instance console log exists: /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.133s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Checking if we can resize image /opt/stack/data/nova/instances/ef4ab4cb-6164-47ea-be1d-cbe0698f991d/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ef4ab4cb-6164-47ea-be1d-cbe0698f991d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.004s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:30:15 user nova-compute[72081]: INFO nova.compute.claims [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Claim successful on node user
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ef4ab4cb-6164-47ea-be1d-cbe0698f991d/disk --force-share --output=json" returned: 0 in 0.154s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Cannot resize image /opt/stack/data/nova/instances/ef4ab4cb-6164-47ea-be1d-cbe0698f991d/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG nova.objects.instance [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lazy-loading 'migration_context' on Instance uuid ef4ab4cb-6164-47ea-be1d-cbe0698f991d {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Ensure instance console log exists: /opt/stack/data/nova/instances/ef4ab4cb-6164-47ea-be1d-cbe0698f991d/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.513s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG nova.compute.manager [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG nova.compute.manager [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG nova.network.neutron [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG nova.network.neutron [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Successfully updated port: 8f7b221a-25ae-41f9-847d-5aedf2d93a71 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:30:16 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:30:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Acquiring lock "refresh_cache-56834002-7593-4592-a86d-23bb90d00be0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Acquired lock "refresh_cache-56834002-7593-4592-a86d-23bb90d00be0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG nova.network.neutron [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG nova.compute.manager [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG nova.compute.manager [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:30:16 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Creating image(s)
Apr 12 03:30:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Acquiring lock "/opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "/opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "/opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.165s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG nova.network.neutron [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Successfully created port: 540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.154s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk 1073741824" returned: 0 in 0.053s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.213s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:16 user nova-compute[72081]: DEBUG nova.policy [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '9405e57e55e84976b0caeba84e3aef62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a027d7708e6747fe970c388d6ba66359', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.179s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Checking if we can resize image /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.compute.manager [req-a39c0f44-c26b-4321-a864-af64109dddf7 req-fabc03ee-de16-4b1a-8403-d30de0f9e35a service nova] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Received event network-changed-8f7b221a-25ae-41f9-847d-5aedf2d93a71 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.compute.manager [req-a39c0f44-c26b-4321-a864-af64109dddf7 req-fabc03ee-de16-4b1a-8403-d30de0f9e35a service nova] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Refreshing instance network info cache due to event network-changed-8f7b221a-25ae-41f9-847d-5aedf2d93a71. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a39c0f44-c26b-4321-a864-af64109dddf7 req-fabc03ee-de16-4b1a-8403-d30de0f9e35a service nova] Acquiring lock "refresh_cache-56834002-7593-4592-a86d-23bb90d00be0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json" returned: 0 in 0.165s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Cannot resize image /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.objects.instance [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lazy-loading 'migration_context' on Instance uuid 72ba1218-065d-40c4-8503-7c4155e47503 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Ensure instance console log exists: /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.network.neutron [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Successfully updated port: dce84390-a2f8-4047-b513-eec5845a2399 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "refresh_cache-2d0904c3-74e0-47bb-a832-758a7035b2f5" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquired lock "refresh_cache-2d0904c3-74e0-47bb-a832-758a7035b2f5" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.network.neutron [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.network.neutron [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Updating instance_info_cache with network_info: [{"id": "8f7b221a-25ae-41f9-847d-5aedf2d93a71", "address": "fa:16:3e:e7:22:1b", "network": {"id": "1a483654-b740-4544-84c9-53a41c11fa88", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1141588183-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "83a7ef3f97fd47849973d4e4482fb5f9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f7b221a-25", "ovs_interfaceid": "8f7b221a-25ae-41f9-847d-5aedf2d93a71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Releasing lock "refresh_cache-56834002-7593-4592-a86d-23bb90d00be0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.compute.manager [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Instance network_info: |[{"id": "8f7b221a-25ae-41f9-847d-5aedf2d93a71", "address": "fa:16:3e:e7:22:1b", "network": {"id": "1a483654-b740-4544-84c9-53a41c11fa88", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1141588183-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "83a7ef3f97fd47849973d4e4482fb5f9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f7b221a-25", "ovs_interfaceid": "8f7b221a-25ae-41f9-847d-5aedf2d93a71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a39c0f44-c26b-4321-a864-af64109dddf7 req-fabc03ee-de16-4b1a-8403-d30de0f9e35a service nova] Acquired lock "refresh_cache-56834002-7593-4592-a86d-23bb90d00be0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.network.neutron [req-a39c0f44-c26b-4321-a864-af64109dddf7 req-fabc03ee-de16-4b1a-8403-d30de0f9e35a service nova] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Refreshing network info cache for port 8f7b221a-25ae-41f9-847d-5aedf2d93a71 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Start _get_guest_xml network_info=[{"id": "8f7b221a-25ae-41f9-847d-5aedf2d93a71", "address": "fa:16:3e:e7:22:1b", "network": {"id": "1a483654-b740-4544-84c9-53a41c11fa88", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1141588183-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "83a7ef3f97fd47849973d4e4482fb5f9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f7b221a-25", "ovs_interfaceid": "8f7b221a-25ae-41f9-847d-5aedf2d93a71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:30:17 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:30:17 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.privsep.utils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72081) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:30:08Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-2007129702',display_name='tempest-ServersNegativeTestJSON-server-2007129702',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-2007129702',id=1,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='83a7ef3f97fd47849973d4e4482fb5f9',ramdisk_id='',reservation_id='r-qls8y0ir',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServersNegativeTestJSON-864113759',owner_user_name='tempest-ServersNegativeTestJSON-864113759-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:30:11Z,user_data=None,user_id='3da7394c51664a7caacfda6b7dda9066',uuid=56834002-7593-4592-a86d-23bb90d00be0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8f7b221a-25ae-41f9-847d-5aedf2d93a71", "address": "fa:16:3e:e7:22:1b", "network": {"id": "1a483654-b740-4544-84c9-53a41c11fa88", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1141588183-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "83a7ef3f97fd47849973d4e4482fb5f9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f7b221a-25", "ovs_interfaceid": "8f7b221a-25ae-41f9-847d-5aedf2d93a71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Converting VIF {"id": "8f7b221a-25ae-41f9-847d-5aedf2d93a71", "address": "fa:16:3e:e7:22:1b", "network": {"id": "1a483654-b740-4544-84c9-53a41c11fa88", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1141588183-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "83a7ef3f97fd47849973d4e4482fb5f9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f7b221a-25", "ovs_interfaceid": "8f7b221a-25ae-41f9-847d-5aedf2d93a71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e7:22:1b,bridge_name='br-int',has_traffic_filtering=True,id=8f7b221a-25ae-41f9-847d-5aedf2d93a71,network=Network(1a483654-b740-4544-84c9-53a41c11fa88),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f7b221a-25') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.objects.instance [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lazy-loading 'pci_devices' on Instance uuid 56834002-7593-4592-a86d-23bb90d00be0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.network.neutron [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:30:17 user nova-compute[72081]:   <uuid>56834002-7593-4592-a86d-23bb90d00be0</uuid>
Apr 12 03:30:17 user nova-compute[72081]:   <name>instance-00000001</name>
Apr 12 03:30:17 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:30:17 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:30:17 user nova-compute[72081]:   <metadata>
Apr 12 03:30:17 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:30:17 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:30:17 user nova-compute[72081]:       <nova:name>tempest-ServersNegativeTestJSON-server-2007129702</nova:name>
Apr 12 03:30:17 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:30:17</nova:creationTime>
Apr 12 03:30:17 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:30:17 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:30:17 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:30:17 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:30:17 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:30:17 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:30:17 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:30:17 user nova-compute[72081]:       <nova:owner>
Apr 12 03:30:17 user nova-compute[72081]:         <nova:user uuid="3da7394c51664a7caacfda6b7dda9066">tempest-ServersNegativeTestJSON-864113759-project-member</nova:user>
Apr 12 03:30:17 user nova-compute[72081]:         <nova:project uuid="83a7ef3f97fd47849973d4e4482fb5f9">tempest-ServersNegativeTestJSON-864113759</nova:project>
Apr 12 03:30:17 user nova-compute[72081]:       </nova:owner>
Apr 12 03:30:17 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:30:17 user nova-compute[72081]:       <nova:ports>
Apr 12 03:30:17 user nova-compute[72081]:         <nova:port uuid="8f7b221a-25ae-41f9-847d-5aedf2d93a71">
Apr 12 03:30:17 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Apr 12 03:30:17 user nova-compute[72081]:         </nova:port>
Apr 12 03:30:17 user nova-compute[72081]:       </nova:ports>
Apr 12 03:30:17 user nova-compute[72081]:     </nova:instance>
Apr 12 03:30:17 user nova-compute[72081]:   </metadata>
Apr 12 03:30:17 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:30:17 user nova-compute[72081]:     <system>
Apr 12 03:30:17 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:30:17 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:30:17 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:30:17 user nova-compute[72081]:       <entry name="serial">56834002-7593-4592-a86d-23bb90d00be0</entry>
Apr 12 03:30:17 user nova-compute[72081]:       <entry name="uuid">56834002-7593-4592-a86d-23bb90d00be0</entry>
Apr 12 03:30:17 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:30:17 user nova-compute[72081]:     </system>
Apr 12 03:30:17 user nova-compute[72081]:   </sysinfo>
Apr 12 03:30:17 user nova-compute[72081]:   <os>
Apr 12 03:30:17 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:30:17 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:30:17 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:30:17 user nova-compute[72081]:   </os>
Apr 12 03:30:17 user nova-compute[72081]:   <features>
Apr 12 03:30:17 user nova-compute[72081]:     <acpi/>
Apr 12 03:30:17 user nova-compute[72081]:     <apic/>
Apr 12 03:30:17 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:30:17 user nova-compute[72081]:   </features>
Apr 12 03:30:17 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:30:17 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:30:17 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:30:17 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:30:17 user nova-compute[72081]:   </clock>
Apr 12 03:30:17 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:30:17 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:30:17 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:30:17 user nova-compute[72081]:   </cpu>
Apr 12 03:30:17 user nova-compute[72081]:   <devices>
Apr 12 03:30:17 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:30:17 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:30:17 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk"/>
Apr 12 03:30:17 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:30:17 user nova-compute[72081]:     </disk>
Apr 12 03:30:17 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:30:17 user nova-compute[72081]:       <mac address="fa:16:3e:e7:22:1b"/>
Apr 12 03:30:17 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:30:17 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:30:17 user nova-compute[72081]:       <target dev="tap8f7b221a-25"/>
Apr 12 03:30:17 user nova-compute[72081]:     </interface>
Apr 12 03:30:17 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:30:17 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/console.log" append="off"/>
Apr 12 03:30:17 user nova-compute[72081]:     </serial>
Apr 12 03:30:17 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:30:17 user nova-compute[72081]:     <video>
Apr 12 03:30:17 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:30:17 user nova-compute[72081]:     </video>
Apr 12 03:30:17 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:30:17 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:30:17 user nova-compute[72081]:     </rng>
Apr 12 03:30:17 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:30:17 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:30:17 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:30:17 user nova-compute[72081]:     </memballoon>
Apr 12 03:30:17 user nova-compute[72081]:   </devices>
Apr 12 03:30:17 user nova-compute[72081]: </domain>
Apr 12 03:30:17 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:30:08Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-2007129702',display_name='tempest-ServersNegativeTestJSON-server-2007129702',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-2007129702',id=1,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='83a7ef3f97fd47849973d4e4482fb5f9',ramdisk_id='',reservation_id='r-qls8y0ir',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServersNegativeTestJSON-864113759',owner_user_name='tempest-ServersNegativeTestJSON-864113759-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:30:11Z,user_data=None,user_id='3da7394c51664a7caacfda6b7dda9066',uuid=56834002-7593-4592-a86d-23bb90d00be0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8f7b221a-25ae-41f9-847d-5aedf2d93a71", "address": "fa:16:3e:e7:22:1b", "network": {"id": "1a483654-b740-4544-84c9-53a41c11fa88", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1141588183-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "83a7ef3f97fd47849973d4e4482fb5f9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f7b221a-25", "ovs_interfaceid": "8f7b221a-25ae-41f9-847d-5aedf2d93a71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Converting VIF {"id": "8f7b221a-25ae-41f9-847d-5aedf2d93a71", "address": "fa:16:3e:e7:22:1b", "network": {"id": "1a483654-b740-4544-84c9-53a41c11fa88", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1141588183-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "83a7ef3f97fd47849973d4e4482fb5f9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f7b221a-25", "ovs_interfaceid": "8f7b221a-25ae-41f9-847d-5aedf2d93a71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e7:22:1b,bridge_name='br-int',has_traffic_filtering=True,id=8f7b221a-25ae-41f9-847d-5aedf2d93a71,network=Network(1a483654-b740-4544-84c9-53a41c11fa88),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f7b221a-25') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:30:17 user nova-compute[72081]: DEBUG os_vif [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:e7:22:1b,bridge_name='br-int',has_traffic_filtering=True,id=8f7b221a-25ae-41f9-847d-5aedf2d93a71,network=Network(1a483654-b740-4544-84c9-53a41c11fa88),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f7b221a-25') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Created schema index Interface.name {{(pid=72081) autocreate_indices /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Created schema index Port.name {{(pid=72081) autocreate_indices /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Created schema index Bridge.name {{(pid=72081) autocreate_indices /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] tcp:127.0.0.1:6640: entering CONNECTING {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [POLLOUT] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:30:18 user nova-compute[72081]: INFO oslo.privsep.daemon [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Running privsep helper: ['sudo', 'nova-rootwrap', '/etc/nova/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/nova/nova-cpu.conf', '--privsep_context', 'vif_plug_ovs.privsep.vif_plug', '--privsep_sock_path', '/tmp/tmpw_3_f0wm/privsep.sock']
Apr 12 03:30:18 user sudo[80901]:    stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/nova-rootwrap /etc/nova/rootwrap.conf privsep-helper --config-file /etc/nova/nova-cpu.conf --privsep_context vif_plug_ovs.privsep.vif_plug --privsep_sock_path /tmp/tmpw_3_f0wm/privsep.sock
Apr 12 03:30:18 user sudo[80901]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1001)
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.compute.manager [req-417d7450-cc07-4705-a961-1463dc760052 req-7cb75786-2d82-44ec-ba73-9b3c838e2c57 service nova] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Received event network-changed-dce84390-a2f8-4047-b513-eec5845a2399 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.compute.manager [req-417d7450-cc07-4705-a961-1463dc760052 req-7cb75786-2d82-44ec-ba73-9b3c838e2c57 service nova] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Refreshing instance network info cache due to event network-changed-dce84390-a2f8-4047-b513-eec5845a2399. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-417d7450-cc07-4705-a961-1463dc760052 req-7cb75786-2d82-44ec-ba73-9b3c838e2c57 service nova] Acquiring lock "refresh_cache-2d0904c3-74e0-47bb-a832-758a7035b2f5" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.network.neutron [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Updating instance_info_cache with network_info: [{"id": "dce84390-a2f8-4047-b513-eec5845a2399", "address": "fa:16:3e:6e:2e:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdce84390-a2", "ovs_interfaceid": "dce84390-a2f8-4047-b513-eec5845a2399", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Releasing lock "refresh_cache-2d0904c3-74e0-47bb-a832-758a7035b2f5" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Instance network_info: |[{"id": "dce84390-a2f8-4047-b513-eec5845a2399", "address": "fa:16:3e:6e:2e:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdce84390-a2", "ovs_interfaceid": "dce84390-a2f8-4047-b513-eec5845a2399", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-417d7450-cc07-4705-a961-1463dc760052 req-7cb75786-2d82-44ec-ba73-9b3c838e2c57 service nova] Acquired lock "refresh_cache-2d0904c3-74e0-47bb-a832-758a7035b2f5" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.network.neutron [req-417d7450-cc07-4705-a961-1463dc760052 req-7cb75786-2d82-44ec-ba73-9b3c838e2c57 service nova] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Refreshing network info cache for port dce84390-a2f8-4047-b513-eec5845a2399 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Start _get_guest_xml network_info=[{"id": "dce84390-a2f8-4047-b513-eec5845a2399", "address": "fa:16:3e:6e:2e:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdce84390-a2", "ovs_interfaceid": "dce84390-a2f8-4047-b513-eec5845a2399", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:30:18 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:30:18 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:30:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1115900831',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1115900831',id=2,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6e1fa352a8c144a1aada7ce0d8818429',ramdisk_id='',reservation_id='r-zgpfox6y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-590302458',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:30:12Z,user_data=None,user_id='d99be569057f432484e4a60b46882bbd',uuid=2d0904c3-74e0-47bb-a832-758a7035b2f5,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "dce84390-a2f8-4047-b513-eec5845a2399", "address": "fa:16:3e:6e:2e:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdce84390-a2", "ovs_interfaceid": "dce84390-a2f8-4047-b513-eec5845a2399", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converting VIF {"id": "dce84390-a2f8-4047-b513-eec5845a2399", "address": "fa:16:3e:6e:2e:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdce84390-a2", "ovs_interfaceid": "dce84390-a2f8-4047-b513-eec5845a2399", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6e:2e:db,bridge_name='br-int',has_traffic_filtering=True,id=dce84390-a2f8-4047-b513-eec5845a2399,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdce84390-a2') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.objects.instance [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lazy-loading 'pci_devices' on Instance uuid 2d0904c3-74e0-47bb-a832-758a7035b2f5 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:30:18 user nova-compute[72081]:   <uuid>2d0904c3-74e0-47bb-a832-758a7035b2f5</uuid>
Apr 12 03:30:18 user nova-compute[72081]:   <name>instance-00000002</name>
Apr 12 03:30:18 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:30:18 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:30:18 user nova-compute[72081]:   <metadata>
Apr 12 03:30:18 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:30:18 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:30:18 user nova-compute[72081]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-1115900831</nova:name>
Apr 12 03:30:18 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:30:18</nova:creationTime>
Apr 12 03:30:18 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:30:18 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:30:18 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:30:18 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:30:18 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:30:18 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:30:18 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:30:18 user nova-compute[72081]:       <nova:owner>
Apr 12 03:30:18 user nova-compute[72081]:         <nova:user uuid="d99be569057f432484e4a60b46882bbd">tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member</nova:user>
Apr 12 03:30:18 user nova-compute[72081]:         <nova:project uuid="6e1fa352a8c144a1aada7ce0d8818429">tempest-ServerBootFromVolumeStableRescueTest-590302458</nova:project>
Apr 12 03:30:18 user nova-compute[72081]:       </nova:owner>
Apr 12 03:30:18 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:30:18 user nova-compute[72081]:       <nova:ports>
Apr 12 03:30:18 user nova-compute[72081]:         <nova:port uuid="dce84390-a2f8-4047-b513-eec5845a2399">
Apr 12 03:30:18 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.11" ipVersion="4"/>
Apr 12 03:30:18 user nova-compute[72081]:         </nova:port>
Apr 12 03:30:18 user nova-compute[72081]:       </nova:ports>
Apr 12 03:30:18 user nova-compute[72081]:     </nova:instance>
Apr 12 03:30:18 user nova-compute[72081]:   </metadata>
Apr 12 03:30:18 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:30:18 user nova-compute[72081]:     <system>
Apr 12 03:30:18 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:30:18 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:30:18 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:30:18 user nova-compute[72081]:       <entry name="serial">2d0904c3-74e0-47bb-a832-758a7035b2f5</entry>
Apr 12 03:30:18 user nova-compute[72081]:       <entry name="uuid">2d0904c3-74e0-47bb-a832-758a7035b2f5</entry>
Apr 12 03:30:18 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:30:18 user nova-compute[72081]:     </system>
Apr 12 03:30:18 user nova-compute[72081]:   </sysinfo>
Apr 12 03:30:18 user nova-compute[72081]:   <os>
Apr 12 03:30:18 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:30:18 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:30:18 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:30:18 user nova-compute[72081]:   </os>
Apr 12 03:30:18 user nova-compute[72081]:   <features>
Apr 12 03:30:18 user nova-compute[72081]:     <acpi/>
Apr 12 03:30:18 user nova-compute[72081]:     <apic/>
Apr 12 03:30:18 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:30:18 user nova-compute[72081]:   </features>
Apr 12 03:30:18 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:30:18 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:30:18 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:30:18 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:30:18 user nova-compute[72081]:   </clock>
Apr 12 03:30:18 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:30:18 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:30:18 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:30:18 user nova-compute[72081]:   </cpu>
Apr 12 03:30:18 user nova-compute[72081]:   <devices>
Apr 12 03:30:18 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:30:18 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:30:18 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk"/>
Apr 12 03:30:18 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:30:18 user nova-compute[72081]:     </disk>
Apr 12 03:30:18 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:30:18 user nova-compute[72081]:       <mac address="fa:16:3e:6e:2e:db"/>
Apr 12 03:30:18 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:30:18 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:30:18 user nova-compute[72081]:       <target dev="tapdce84390-a2"/>
Apr 12 03:30:18 user nova-compute[72081]:     </interface>
Apr 12 03:30:18 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:30:18 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/console.log" append="off"/>
Apr 12 03:30:18 user nova-compute[72081]:     </serial>
Apr 12 03:30:18 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:30:18 user nova-compute[72081]:     <video>
Apr 12 03:30:18 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:30:18 user nova-compute[72081]:     </video>
Apr 12 03:30:18 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:30:18 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:30:18 user nova-compute[72081]:     </rng>
Apr 12 03:30:18 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:30:18 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:30:18 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:30:18 user nova-compute[72081]:     </memballoon>
Apr 12 03:30:18 user nova-compute[72081]:   </devices>
Apr 12 03:30:18 user nova-compute[72081]: </domain>
Apr 12 03:30:18 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:30:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1115900831',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1115900831',id=2,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6e1fa352a8c144a1aada7ce0d8818429',ramdisk_id='',reservation_id='r-zgpfox6y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-590302458',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:30:12Z,user_data=None,user_id='d99be569057f432484e4a60b46882bbd',uuid=2d0904c3-74e0-47bb-a832-758a7035b2f5,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "dce84390-a2f8-4047-b513-eec5845a2399", "address": "fa:16:3e:6e:2e:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdce84390-a2", "ovs_interfaceid": "dce84390-a2f8-4047-b513-eec5845a2399", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converting VIF {"id": "dce84390-a2f8-4047-b513-eec5845a2399", "address": "fa:16:3e:6e:2e:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdce84390-a2", "ovs_interfaceid": "dce84390-a2f8-4047-b513-eec5845a2399", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6e:2e:db,bridge_name='br-int',has_traffic_filtering=True,id=dce84390-a2f8-4047-b513-eec5845a2399,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdce84390-a2') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG os_vif [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:6e:2e:db,bridge_name='br-int',has_traffic_filtering=True,id=dce84390-a2f8-4047-b513-eec5845a2399,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdce84390-a2') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:30:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:30:19 user nova-compute[72081]: DEBUG nova.network.neutron [req-a39c0f44-c26b-4321-a864-af64109dddf7 req-fabc03ee-de16-4b1a-8403-d30de0f9e35a service nova] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Updated VIF entry in instance network info cache for port 8f7b221a-25ae-41f9-847d-5aedf2d93a71. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:30:19 user nova-compute[72081]: DEBUG nova.network.neutron [req-a39c0f44-c26b-4321-a864-af64109dddf7 req-fabc03ee-de16-4b1a-8403-d30de0f9e35a service nova] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Updating instance_info_cache with network_info: [{"id": "8f7b221a-25ae-41f9-847d-5aedf2d93a71", "address": "fa:16:3e:e7:22:1b", "network": {"id": "1a483654-b740-4544-84c9-53a41c11fa88", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1141588183-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "83a7ef3f97fd47849973d4e4482fb5f9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f7b221a-25", "ovs_interfaceid": "8f7b221a-25ae-41f9-847d-5aedf2d93a71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:30:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a39c0f44-c26b-4321-a864-af64109dddf7 req-fabc03ee-de16-4b1a-8403-d30de0f9e35a service nova] Releasing lock "refresh_cache-56834002-7593-4592-a86d-23bb90d00be0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:30:19 user nova-compute[72081]: DEBUG nova.network.neutron [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Successfully created port: cc0a860c-4dc1-4a09-836a-3878bcb30d40 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:30:19 user nova-compute[72081]: DEBUG nova.network.neutron [req-417d7450-cc07-4705-a961-1463dc760052 req-7cb75786-2d82-44ec-ba73-9b3c838e2c57 service nova] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Updated VIF entry in instance network info cache for port dce84390-a2f8-4047-b513-eec5845a2399. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:30:19 user nova-compute[72081]: DEBUG nova.network.neutron [req-417d7450-cc07-4705-a961-1463dc760052 req-7cb75786-2d82-44ec-ba73-9b3c838e2c57 service nova] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Updating instance_info_cache with network_info: [{"id": "dce84390-a2f8-4047-b513-eec5845a2399", "address": "fa:16:3e:6e:2e:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdce84390-a2", "ovs_interfaceid": "dce84390-a2f8-4047-b513-eec5845a2399", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:30:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-417d7450-cc07-4705-a961-1463dc760052 req-7cb75786-2d82-44ec-ba73-9b3c838e2c57 service nova] Releasing lock "refresh_cache-2d0904c3-74e0-47bb-a832-758a7035b2f5" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:30:19 user sudo[80901]: pam_unix(sudo:session): session closed for user root
Apr 12 03:30:19 user nova-compute[72081]: INFO oslo.privsep.daemon [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Spawned new privsep daemon via rootwrap
Apr 12 03:30:19 user nova-compute[72081]: INFO oslo.privsep.daemon [-] privsep daemon starting
Apr 12 03:30:19 user nova-compute[72081]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
Apr 12 03:30:19 user nova-compute[72081]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_DAC_OVERRIDE|CAP_NET_ADMIN/CAP_DAC_OVERRIDE|CAP_NET_ADMIN/none
Apr 12 03:30:19 user nova-compute[72081]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 80928
Apr 12 03:30:19 user nova-compute[72081]: WARNING oslo_privsep.priv_context [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] privsep daemon already running
Apr 12 03:30:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapdce84390-a2, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:30:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapdce84390-a2, col_values=(('external_ids', {'iface-id': 'dce84390-a2f8-4047-b513-eec5845a2399', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:6e:2e:db', 'vm-uuid': '2d0904c3-74e0-47bb-a832-758a7035b2f5'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:30:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:30:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:20 user nova-compute[72081]: INFO os_vif [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:6e:2e:db,bridge_name='br-int',has_traffic_filtering=True,id=dce84390-a2f8-4047-b513-eec5845a2399,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdce84390-a2')
Apr 12 03:30:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap8f7b221a-25, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:30:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap8f7b221a-25, col_values=(('external_ids', {'iface-id': '8f7b221a-25ae-41f9-847d-5aedf2d93a71', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:e7:22:1b', 'vm-uuid': '56834002-7593-4592-a86d-23bb90d00be0'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:30:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:30:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:20 user nova-compute[72081]: INFO os_vif [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:e7:22:1b,bridge_name='br-int',has_traffic_filtering=True,id=8f7b221a-25ae-41f9-847d-5aedf2d93a71,network=Network(1a483654-b740-4544-84c9-53a41c11fa88),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f7b221a-25')
Apr 12 03:30:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:30:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] No VIF found with MAC fa:16:3e:6e:2e:db, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:30:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:30:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] No VIF found with MAC fa:16:3e:e7:22:1b, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:30:21 user nova-compute[72081]: DEBUG nova.network.neutron [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Successfully created port: cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:30:21 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Successfully updated port: 540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:30:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "refresh_cache-b23d4bcd-989a-45e1-89a3-6e3d30229a32" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:30:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquired lock "refresh_cache-b23d4bcd-989a-45e1-89a3-6e3d30229a32" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:30:21 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:30:21 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:30:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:22 user nova-compute[72081]: DEBUG nova.compute.manager [req-8c1f7106-37b9-46d3-bc78-37014ea1c0a6 req-8349ec48-3f1c-45d6-8c69-5576d270c4b9 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Received event network-changed-540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:30:22 user nova-compute[72081]: DEBUG nova.compute.manager [req-8c1f7106-37b9-46d3-bc78-37014ea1c0a6 req-8349ec48-3f1c-45d6-8c69-5576d270c4b9 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Refreshing instance network info cache due to event network-changed-540b7307-8fea-4b50-bab8-626983135a96. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:30:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8c1f7106-37b9-46d3-bc78-37014ea1c0a6 req-8349ec48-3f1c-45d6-8c69-5576d270c4b9 service nova] Acquiring lock "refresh_cache-b23d4bcd-989a-45e1-89a3-6e3d30229a32" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:30:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Updating instance_info_cache with network_info: [{"id": "540b7307-8fea-4b50-bab8-626983135a96", "address": "fa:16:3e:8d:a9:ab", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap540b7307-8f", "ovs_interfaceid": "540b7307-8fea-4b50-bab8-626983135a96", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Releasing lock "refresh_cache-b23d4bcd-989a-45e1-89a3-6e3d30229a32" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Instance network_info: |[{"id": "540b7307-8fea-4b50-bab8-626983135a96", "address": "fa:16:3e:8d:a9:ab", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap540b7307-8f", "ovs_interfaceid": "540b7307-8fea-4b50-bab8-626983135a96", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8c1f7106-37b9-46d3-bc78-37014ea1c0a6 req-8349ec48-3f1c-45d6-8c69-5576d270c4b9 service nova] Acquired lock "refresh_cache-b23d4bcd-989a-45e1-89a3-6e3d30229a32" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.network.neutron [req-8c1f7106-37b9-46d3-bc78-37014ea1c0a6 req-8349ec48-3f1c-45d6-8c69-5576d270c4b9 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Refreshing network info cache for port 540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Start _get_guest_xml network_info=[{"id": "540b7307-8fea-4b50-bab8-626983135a96", "address": "fa:16:3e:8d:a9:ab", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap540b7307-8f", "ovs_interfaceid": "540b7307-8fea-4b50-bab8-626983135a96", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:30:23 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:30:23 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:30:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-1925048093',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-1925048093',id=3,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBNv3aMnfqZBrraTWMhLhVcBPnIqd/pyo9633J0szzOpRjklWBRcLk+TXv631ENNLeAHi6cKzIvn2qCEpBSN9HX63yWj5klT0kBwkXMpaNwu0eHxsmOThEKjvkK+5wRQIg==',key_name='tempest-keypair-892402082',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='738ebb6b798c4974aa46f9bbe3828279',ramdisk_id='',reservation_id='r-ybseqfps',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeShelveTestJSON-2139975165',owner_user_name='tempest-AttachVolumeShelveTestJSON-2139975165-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:30:12Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='de1fe5db01624224ab7b615c908d990b',uuid=b23d4bcd-989a-45e1-89a3-6e3d30229a32,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "540b7307-8fea-4b50-bab8-626983135a96", "address": "fa:16:3e:8d:a9:ab", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap540b7307-8f", "ovs_interfaceid": "540b7307-8fea-4b50-bab8-626983135a96", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Converting VIF {"id": "540b7307-8fea-4b50-bab8-626983135a96", "address": "fa:16:3e:8d:a9:ab", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap540b7307-8f", "ovs_interfaceid": "540b7307-8fea-4b50-bab8-626983135a96", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:8d:a9:ab,bridge_name='br-int',has_traffic_filtering=True,id=540b7307-8fea-4b50-bab8-626983135a96,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap540b7307-8f') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.objects.instance [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lazy-loading 'pci_devices' on Instance uuid b23d4bcd-989a-45e1-89a3-6e3d30229a32 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:30:23 user nova-compute[72081]:   <uuid>b23d4bcd-989a-45e1-89a3-6e3d30229a32</uuid>
Apr 12 03:30:23 user nova-compute[72081]:   <name>instance-00000003</name>
Apr 12 03:30:23 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:30:23 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:30:23 user nova-compute[72081]:   <metadata>
Apr 12 03:30:23 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:30:23 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:30:23 user nova-compute[72081]:       <nova:name>tempest-AttachVolumeShelveTestJSON-server-1925048093</nova:name>
Apr 12 03:30:23 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:30:23</nova:creationTime>
Apr 12 03:30:23 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:30:23 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:30:23 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:30:23 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:30:23 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:30:23 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:30:23 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:30:23 user nova-compute[72081]:       <nova:owner>
Apr 12 03:30:23 user nova-compute[72081]:         <nova:user uuid="de1fe5db01624224ab7b615c908d990b">tempest-AttachVolumeShelveTestJSON-2139975165-project-member</nova:user>
Apr 12 03:30:23 user nova-compute[72081]:         <nova:project uuid="738ebb6b798c4974aa46f9bbe3828279">tempest-AttachVolumeShelveTestJSON-2139975165</nova:project>
Apr 12 03:30:23 user nova-compute[72081]:       </nova:owner>
Apr 12 03:30:23 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:30:23 user nova-compute[72081]:       <nova:ports>
Apr 12 03:30:23 user nova-compute[72081]:         <nova:port uuid="540b7307-8fea-4b50-bab8-626983135a96">
Apr 12 03:30:23 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Apr 12 03:30:23 user nova-compute[72081]:         </nova:port>
Apr 12 03:30:23 user nova-compute[72081]:       </nova:ports>
Apr 12 03:30:23 user nova-compute[72081]:     </nova:instance>
Apr 12 03:30:23 user nova-compute[72081]:   </metadata>
Apr 12 03:30:23 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:30:23 user nova-compute[72081]:     <system>
Apr 12 03:30:23 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:30:23 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:30:23 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:30:23 user nova-compute[72081]:       <entry name="serial">b23d4bcd-989a-45e1-89a3-6e3d30229a32</entry>
Apr 12 03:30:23 user nova-compute[72081]:       <entry name="uuid">b23d4bcd-989a-45e1-89a3-6e3d30229a32</entry>
Apr 12 03:30:23 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:30:23 user nova-compute[72081]:     </system>
Apr 12 03:30:23 user nova-compute[72081]:   </sysinfo>
Apr 12 03:30:23 user nova-compute[72081]:   <os>
Apr 12 03:30:23 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:30:23 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:30:23 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:30:23 user nova-compute[72081]:   </os>
Apr 12 03:30:23 user nova-compute[72081]:   <features>
Apr 12 03:30:23 user nova-compute[72081]:     <acpi/>
Apr 12 03:30:23 user nova-compute[72081]:     <apic/>
Apr 12 03:30:23 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:30:23 user nova-compute[72081]:   </features>
Apr 12 03:30:23 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:30:23 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:30:23 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:30:23 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:30:23 user nova-compute[72081]:   </clock>
Apr 12 03:30:23 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:30:23 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:30:23 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:30:23 user nova-compute[72081]:   </cpu>
Apr 12 03:30:23 user nova-compute[72081]:   <devices>
Apr 12 03:30:23 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:30:23 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:30:23 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk"/>
Apr 12 03:30:23 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:30:23 user nova-compute[72081]:     </disk>
Apr 12 03:30:23 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:30:23 user nova-compute[72081]:       <mac address="fa:16:3e:8d:a9:ab"/>
Apr 12 03:30:23 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:30:23 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:30:23 user nova-compute[72081]:       <target dev="tap540b7307-8f"/>
Apr 12 03:30:23 user nova-compute[72081]:     </interface>
Apr 12 03:30:23 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:30:23 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/console.log" append="off"/>
Apr 12 03:30:23 user nova-compute[72081]:     </serial>
Apr 12 03:30:23 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:30:23 user nova-compute[72081]:     <video>
Apr 12 03:30:23 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:30:23 user nova-compute[72081]:     </video>
Apr 12 03:30:23 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:30:23 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:30:23 user nova-compute[72081]:     </rng>
Apr 12 03:30:23 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:30:23 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:30:23 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:30:23 user nova-compute[72081]:     </memballoon>
Apr 12 03:30:23 user nova-compute[72081]:   </devices>
Apr 12 03:30:23 user nova-compute[72081]: </domain>
Apr 12 03:30:23 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:30:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-1925048093',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-1925048093',id=3,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBNv3aMnfqZBrraTWMhLhVcBPnIqd/pyo9633J0szzOpRjklWBRcLk+TXv631ENNLeAHi6cKzIvn2qCEpBSN9HX63yWj5klT0kBwkXMpaNwu0eHxsmOThEKjvkK+5wRQIg==',key_name='tempest-keypair-892402082',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='738ebb6b798c4974aa46f9bbe3828279',ramdisk_id='',reservation_id='r-ybseqfps',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeShelveTestJSON-2139975165',owner_user_name='tempest-AttachVolumeShelveTestJSON-2139975165-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:30:12Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='de1fe5db01624224ab7b615c908d990b',uuid=b23d4bcd-989a-45e1-89a3-6e3d30229a32,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "540b7307-8fea-4b50-bab8-626983135a96", "address": "fa:16:3e:8d:a9:ab", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap540b7307-8f", "ovs_interfaceid": "540b7307-8fea-4b50-bab8-626983135a96", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Converting VIF {"id": "540b7307-8fea-4b50-bab8-626983135a96", "address": "fa:16:3e:8d:a9:ab", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap540b7307-8f", "ovs_interfaceid": "540b7307-8fea-4b50-bab8-626983135a96", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:8d:a9:ab,bridge_name='br-int',has_traffic_filtering=True,id=540b7307-8fea-4b50-bab8-626983135a96,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap540b7307-8f') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG os_vif [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:8d:a9:ab,bridge_name='br-int',has_traffic_filtering=True,id=540b7307-8fea-4b50-bab8-626983135a96,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap540b7307-8f') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap540b7307-8f, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap540b7307-8f, col_values=(('external_ids', {'iface-id': '540b7307-8fea-4b50-bab8-626983135a96', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:8d:a9:ab', 'vm-uuid': 'b23d4bcd-989a-45e1-89a3-6e3d30229a32'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:23 user nova-compute[72081]: INFO os_vif [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:8d:a9:ab,bridge_name='br-int',has_traffic_filtering=True,id=540b7307-8fea-4b50-bab8-626983135a96,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap540b7307-8f')
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.network.neutron [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Successfully updated port: cc0a860c-4dc1-4a09-836a-3878bcb30d40 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Acquiring lock "refresh_cache-ef4ab4cb-6164-47ea-be1d-cbe0698f991d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Acquired lock "refresh_cache-ef4ab4cb-6164-47ea-be1d-cbe0698f991d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.network.neutron [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] No VIF found with MAC fa:16:3e:8d:a9:ab, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.compute.manager [req-d53f0fe1-d6ea-4e8a-b8bc-884c8ea1c4e5 req-6a5c0af7-9c31-461d-b352-20b80fb3b3c5 service nova] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Received event network-vif-plugged-dce84390-a2f8-4047-b513-eec5845a2399 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d53f0fe1-d6ea-4e8a-b8bc-884c8ea1c4e5 req-6a5c0af7-9c31-461d-b352-20b80fb3b3c5 service nova] Acquiring lock "2d0904c3-74e0-47bb-a832-758a7035b2f5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d53f0fe1-d6ea-4e8a-b8bc-884c8ea1c4e5 req-6a5c0af7-9c31-461d-b352-20b80fb3b3c5 service nova] Lock "2d0904c3-74e0-47bb-a832-758a7035b2f5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d53f0fe1-d6ea-4e8a-b8bc-884c8ea1c4e5 req-6a5c0af7-9c31-461d-b352-20b80fb3b3c5 service nova] Lock "2d0904c3-74e0-47bb-a832-758a7035b2f5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:23 user nova-compute[72081]: DEBUG nova.compute.manager [req-d53f0fe1-d6ea-4e8a-b8bc-884c8ea1c4e5 req-6a5c0af7-9c31-461d-b352-20b80fb3b3c5 service nova] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] No waiting events found dispatching network-vif-plugged-dce84390-a2f8-4047-b513-eec5845a2399 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:30:23 user nova-compute[72081]: WARNING nova.compute.manager [req-d53f0fe1-d6ea-4e8a-b8bc-884c8ea1c4e5 req-6a5c0af7-9c31-461d-b352-20b80fb3b3c5 service nova] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Received unexpected event network-vif-plugged-dce84390-a2f8-4047-b513-eec5845a2399 for instance with vm_state building and task_state spawning.
Apr 12 03:30:24 user nova-compute[72081]: DEBUG nova.network.neutron [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:30:24 user nova-compute[72081]: DEBUG nova.compute.manager [req-4a2e4b53-2ab1-4294-89d9-a5f7970f130c req-bd0d1c92-0410-467f-904f-f13a38b80d29 service nova] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Received event network-changed-cc0a860c-4dc1-4a09-836a-3878bcb30d40 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:30:24 user nova-compute[72081]: DEBUG nova.compute.manager [req-4a2e4b53-2ab1-4294-89d9-a5f7970f130c req-bd0d1c92-0410-467f-904f-f13a38b80d29 service nova] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Refreshing instance network info cache due to event network-changed-cc0a860c-4dc1-4a09-836a-3878bcb30d40. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:30:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4a2e4b53-2ab1-4294-89d9-a5f7970f130c req-bd0d1c92-0410-467f-904f-f13a38b80d29 service nova] Acquiring lock "refresh_cache-ef4ab4cb-6164-47ea-be1d-cbe0698f991d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:30:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:25 user nova-compute[72081]: DEBUG nova.network.neutron [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Successfully updated port: cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.network.neutron [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Updating instance_info_cache with network_info: [{"id": "cc0a860c-4dc1-4a09-836a-3878bcb30d40", "address": "fa:16:3e:ac:8c:49", "network": {"id": "0f8e4d8d-8d90-47de-a965-5610331236ef", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-2097572913-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5dadbf1b0a124d2d929c85bc2b916022", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcc0a860c-4d", "ovs_interfaceid": "cc0a860c-4dc1-4a09-836a-3878bcb30d40", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Acquiring lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Acquired lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.network.neutron [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Releasing lock "refresh_cache-ef4ab4cb-6164-47ea-be1d-cbe0698f991d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.compute.manager [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Instance network_info: |[{"id": "cc0a860c-4dc1-4a09-836a-3878bcb30d40", "address": "fa:16:3e:ac:8c:49", "network": {"id": "0f8e4d8d-8d90-47de-a965-5610331236ef", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-2097572913-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5dadbf1b0a124d2d929c85bc2b916022", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcc0a860c-4d", "ovs_interfaceid": "cc0a860c-4dc1-4a09-836a-3878bcb30d40", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4a2e4b53-2ab1-4294-89d9-a5f7970f130c req-bd0d1c92-0410-467f-904f-f13a38b80d29 service nova] Acquired lock "refresh_cache-ef4ab4cb-6164-47ea-be1d-cbe0698f991d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.network.neutron [req-4a2e4b53-2ab1-4294-89d9-a5f7970f130c req-bd0d1c92-0410-467f-904f-f13a38b80d29 service nova] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Refreshing network info cache for port cc0a860c-4dc1-4a09-836a-3878bcb30d40 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Start _get_guest_xml network_info=[{"id": "cc0a860c-4dc1-4a09-836a-3878bcb30d40", "address": "fa:16:3e:ac:8c:49", "network": {"id": "0f8e4d8d-8d90-47de-a965-5610331236ef", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-2097572913-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5dadbf1b0a124d2d929c85bc2b916022", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcc0a860c-4d", "ovs_interfaceid": "cc0a860c-4dc1-4a09-836a-3878bcb30d40", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-cd5fdf31-7a62-4a03-95ce-8f39583592ef req-8258b9c9-f691-40fe-baa3-ef3ac6cd3091 service nova] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Received event network-vif-plugged-dce84390-a2f8-4047-b513-eec5845a2399 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cd5fdf31-7a62-4a03-95ce-8f39583592ef req-8258b9c9-f691-40fe-baa3-ef3ac6cd3091 service nova] Acquiring lock "2d0904c3-74e0-47bb-a832-758a7035b2f5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cd5fdf31-7a62-4a03-95ce-8f39583592ef req-8258b9c9-f691-40fe-baa3-ef3ac6cd3091 service nova] Lock "2d0904c3-74e0-47bb-a832-758a7035b2f5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cd5fdf31-7a62-4a03-95ce-8f39583592ef req-8258b9c9-f691-40fe-baa3-ef3ac6cd3091 service nova] Lock "2d0904c3-74e0-47bb-a832-758a7035b2f5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-cd5fdf31-7a62-4a03-95ce-8f39583592ef req-8258b9c9-f691-40fe-baa3-ef3ac6cd3091 service nova] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] No waiting events found dispatching network-vif-plugged-dce84390-a2f8-4047-b513-eec5845a2399 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:30:26 user nova-compute[72081]: WARNING nova.compute.manager [req-cd5fdf31-7a62-4a03-95ce-8f39583592ef req-8258b9c9-f691-40fe-baa3-ef3ac6cd3091 service nova] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Received unexpected event network-vif-plugged-dce84390-a2f8-4047-b513-eec5845a2399 for instance with vm_state building and task_state spawning.
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-cd5fdf31-7a62-4a03-95ce-8f39583592ef req-8258b9c9-f691-40fe-baa3-ef3ac6cd3091 service nova] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Received event network-vif-plugged-8f7b221a-25ae-41f9-847d-5aedf2d93a71 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cd5fdf31-7a62-4a03-95ce-8f39583592ef req-8258b9c9-f691-40fe-baa3-ef3ac6cd3091 service nova] Acquiring lock "56834002-7593-4592-a86d-23bb90d00be0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cd5fdf31-7a62-4a03-95ce-8f39583592ef req-8258b9c9-f691-40fe-baa3-ef3ac6cd3091 service nova] Lock "56834002-7593-4592-a86d-23bb90d00be0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cd5fdf31-7a62-4a03-95ce-8f39583592ef req-8258b9c9-f691-40fe-baa3-ef3ac6cd3091 service nova] Lock "56834002-7593-4592-a86d-23bb90d00be0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-cd5fdf31-7a62-4a03-95ce-8f39583592ef req-8258b9c9-f691-40fe-baa3-ef3ac6cd3091 service nova] [instance: 56834002-7593-4592-a86d-23bb90d00be0] No waiting events found dispatching network-vif-plugged-8f7b221a-25ae-41f9-847d-5aedf2d93a71 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:30:26 user nova-compute[72081]: WARNING nova.compute.manager [req-cd5fdf31-7a62-4a03-95ce-8f39583592ef req-8258b9c9-f691-40fe-baa3-ef3ac6cd3091 service nova] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Received unexpected event network-vif-plugged-8f7b221a-25ae-41f9-847d-5aedf2d93a71 for instance with vm_state building and task_state spawning.
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-cd5fdf31-7a62-4a03-95ce-8f39583592ef req-8258b9c9-f691-40fe-baa3-ef3ac6cd3091 service nova] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Received event network-vif-plugged-8f7b221a-25ae-41f9-847d-5aedf2d93a71 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cd5fdf31-7a62-4a03-95ce-8f39583592ef req-8258b9c9-f691-40fe-baa3-ef3ac6cd3091 service nova] Acquiring lock "56834002-7593-4592-a86d-23bb90d00be0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cd5fdf31-7a62-4a03-95ce-8f39583592ef req-8258b9c9-f691-40fe-baa3-ef3ac6cd3091 service nova] Lock "56834002-7593-4592-a86d-23bb90d00be0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cd5fdf31-7a62-4a03-95ce-8f39583592ef req-8258b9c9-f691-40fe-baa3-ef3ac6cd3091 service nova] Lock "56834002-7593-4592-a86d-23bb90d00be0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-cd5fdf31-7a62-4a03-95ce-8f39583592ef req-8258b9c9-f691-40fe-baa3-ef3ac6cd3091 service nova] [instance: 56834002-7593-4592-a86d-23bb90d00be0] No waiting events found dispatching network-vif-plugged-8f7b221a-25ae-41f9-847d-5aedf2d93a71 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:30:26 user nova-compute[72081]: WARNING nova.compute.manager [req-cd5fdf31-7a62-4a03-95ce-8f39583592ef req-8258b9c9-f691-40fe-baa3-ef3ac6cd3091 service nova] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Received unexpected event network-vif-plugged-8f7b221a-25ae-41f9-847d-5aedf2d93a71 for instance with vm_state building and task_state spawning.
Apr 12 03:30:26 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:30:26 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:30:13Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-DeleteServersTestJSON-server-160677355',display_name='tempest-DeleteServersTestJSON-server-160677355',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-deleteserverstestjson-server-160677355',id=4,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='5dadbf1b0a124d2d929c85bc2b916022',ramdisk_id='',reservation_id='r-auk5xva2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-DeleteServersTestJSON-78784513',owner_user_name='tempest-DeleteServersTestJSON-78784513-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:30:15Z,user_data=None,user_id='4d01e7aaa3734b299d0c0a27844011b4',uuid=ef4ab4cb-6164-47ea-be1d-cbe0698f991d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "cc0a860c-4dc1-4a09-836a-3878bcb30d40", "address": "fa:16:3e:ac:8c:49", "network": {"id": "0f8e4d8d-8d90-47de-a965-5610331236ef", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-2097572913-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5dadbf1b0a124d2d929c85bc2b916022", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcc0a860c-4d", "ovs_interfaceid": "cc0a860c-4dc1-4a09-836a-3878bcb30d40", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Converting VIF {"id": "cc0a860c-4dc1-4a09-836a-3878bcb30d40", "address": "fa:16:3e:ac:8c:49", "network": {"id": "0f8e4d8d-8d90-47de-a965-5610331236ef", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-2097572913-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5dadbf1b0a124d2d929c85bc2b916022", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcc0a860c-4d", "ovs_interfaceid": "cc0a860c-4dc1-4a09-836a-3878bcb30d40", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ac:8c:49,bridge_name='br-int',has_traffic_filtering=True,id=cc0a860c-4dc1-4a09-836a-3878bcb30d40,network=Network(0f8e4d8d-8d90-47de-a965-5610331236ef),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapcc0a860c-4d') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.objects.instance [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lazy-loading 'pci_devices' on Instance uuid ef4ab4cb-6164-47ea-be1d-cbe0698f991d {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.network.neutron [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:30:26 user nova-compute[72081]:   <uuid>ef4ab4cb-6164-47ea-be1d-cbe0698f991d</uuid>
Apr 12 03:30:26 user nova-compute[72081]:   <name>instance-00000004</name>
Apr 12 03:30:26 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:30:26 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:30:26 user nova-compute[72081]:   <metadata>
Apr 12 03:30:26 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:30:26 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:30:26 user nova-compute[72081]:       <nova:name>tempest-DeleteServersTestJSON-server-160677355</nova:name>
Apr 12 03:30:26 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:30:26</nova:creationTime>
Apr 12 03:30:26 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:30:26 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:30:26 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:30:26 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:30:26 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:30:26 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:30:26 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:30:26 user nova-compute[72081]:       <nova:owner>
Apr 12 03:30:26 user nova-compute[72081]:         <nova:user uuid="4d01e7aaa3734b299d0c0a27844011b4">tempest-DeleteServersTestJSON-78784513-project-member</nova:user>
Apr 12 03:30:26 user nova-compute[72081]:         <nova:project uuid="5dadbf1b0a124d2d929c85bc2b916022">tempest-DeleteServersTestJSON-78784513</nova:project>
Apr 12 03:30:26 user nova-compute[72081]:       </nova:owner>
Apr 12 03:30:26 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:30:26 user nova-compute[72081]:       <nova:ports>
Apr 12 03:30:26 user nova-compute[72081]:         <nova:port uuid="cc0a860c-4dc1-4a09-836a-3878bcb30d40">
Apr 12 03:30:26 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.11" ipVersion="4"/>
Apr 12 03:30:26 user nova-compute[72081]:         </nova:port>
Apr 12 03:30:26 user nova-compute[72081]:       </nova:ports>
Apr 12 03:30:26 user nova-compute[72081]:     </nova:instance>
Apr 12 03:30:26 user nova-compute[72081]:   </metadata>
Apr 12 03:30:26 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:30:26 user nova-compute[72081]:     <system>
Apr 12 03:30:26 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:30:26 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:30:26 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:30:26 user nova-compute[72081]:       <entry name="serial">ef4ab4cb-6164-47ea-be1d-cbe0698f991d</entry>
Apr 12 03:30:26 user nova-compute[72081]:       <entry name="uuid">ef4ab4cb-6164-47ea-be1d-cbe0698f991d</entry>
Apr 12 03:30:26 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:30:26 user nova-compute[72081]:     </system>
Apr 12 03:30:26 user nova-compute[72081]:   </sysinfo>
Apr 12 03:30:26 user nova-compute[72081]:   <os>
Apr 12 03:30:26 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:30:26 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:30:26 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:30:26 user nova-compute[72081]:   </os>
Apr 12 03:30:26 user nova-compute[72081]:   <features>
Apr 12 03:30:26 user nova-compute[72081]:     <acpi/>
Apr 12 03:30:26 user nova-compute[72081]:     <apic/>
Apr 12 03:30:26 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:30:26 user nova-compute[72081]:   </features>
Apr 12 03:30:26 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:30:26 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:30:26 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:30:26 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:30:26 user nova-compute[72081]:   </clock>
Apr 12 03:30:26 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:30:26 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:30:26 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:30:26 user nova-compute[72081]:   </cpu>
Apr 12 03:30:26 user nova-compute[72081]:   <devices>
Apr 12 03:30:26 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:30:26 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:30:26 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/ef4ab4cb-6164-47ea-be1d-cbe0698f991d/disk"/>
Apr 12 03:30:26 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:30:26 user nova-compute[72081]:     </disk>
Apr 12 03:30:26 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:30:26 user nova-compute[72081]:       <mac address="fa:16:3e:ac:8c:49"/>
Apr 12 03:30:26 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:30:26 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:30:26 user nova-compute[72081]:       <target dev="tapcc0a860c-4d"/>
Apr 12 03:30:26 user nova-compute[72081]:     </interface>
Apr 12 03:30:26 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:30:26 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/ef4ab4cb-6164-47ea-be1d-cbe0698f991d/console.log" append="off"/>
Apr 12 03:30:26 user nova-compute[72081]:     </serial>
Apr 12 03:30:26 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:30:26 user nova-compute[72081]:     <video>
Apr 12 03:30:26 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:30:26 user nova-compute[72081]:     </video>
Apr 12 03:30:26 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:30:26 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:30:26 user nova-compute[72081]:     </rng>
Apr 12 03:30:26 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:30:26 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:30:26 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:30:26 user nova-compute[72081]:     </memballoon>
Apr 12 03:30:26 user nova-compute[72081]:   </devices>
Apr 12 03:30:26 user nova-compute[72081]: </domain>
Apr 12 03:30:26 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:30:13Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-DeleteServersTestJSON-server-160677355',display_name='tempest-DeleteServersTestJSON-server-160677355',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-deleteserverstestjson-server-160677355',id=4,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='5dadbf1b0a124d2d929c85bc2b916022',ramdisk_id='',reservation_id='r-auk5xva2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-DeleteServersTestJSON-78784513',owner_user_name='tempest-DeleteServersTestJSON-78784513-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:30:15Z,user_data=None,user_id='4d01e7aaa3734b299d0c0a27844011b4',uuid=ef4ab4cb-6164-47ea-be1d-cbe0698f991d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "cc0a860c-4dc1-4a09-836a-3878bcb30d40", "address": "fa:16:3e:ac:8c:49", "network": {"id": "0f8e4d8d-8d90-47de-a965-5610331236ef", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-2097572913-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5dadbf1b0a124d2d929c85bc2b916022", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcc0a860c-4d", "ovs_interfaceid": "cc0a860c-4dc1-4a09-836a-3878bcb30d40", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Converting VIF {"id": "cc0a860c-4dc1-4a09-836a-3878bcb30d40", "address": "fa:16:3e:ac:8c:49", "network": {"id": "0f8e4d8d-8d90-47de-a965-5610331236ef", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-2097572913-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5dadbf1b0a124d2d929c85bc2b916022", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcc0a860c-4d", "ovs_interfaceid": "cc0a860c-4dc1-4a09-836a-3878bcb30d40", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ac:8c:49,bridge_name='br-int',has_traffic_filtering=True,id=cc0a860c-4dc1-4a09-836a-3878bcb30d40,network=Network(0f8e4d8d-8d90-47de-a965-5610331236ef),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapcc0a860c-4d') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG os_vif [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:ac:8c:49,bridge_name='br-int',has_traffic_filtering=True,id=cc0a860c-4dc1-4a09-836a-3878bcb30d40,network=Network(0f8e4d8d-8d90-47de-a965-5610331236ef),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapcc0a860c-4d') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapcc0a860c-4d, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapcc0a860c-4d, col_values=(('external_ids', {'iface-id': 'cc0a860c-4dc1-4a09-836a-3878bcb30d40', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:ac:8c:49', 'vm-uuid': 'ef4ab4cb-6164-47ea-be1d-cbe0698f991d'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:26 user nova-compute[72081]: INFO os_vif [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:ac:8c:49,bridge_name='br-int',has_traffic_filtering=True,id=cc0a860c-4dc1-4a09-836a-3878bcb30d40,network=Network(0f8e4d8d-8d90-47de-a965-5610331236ef),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapcc0a860c-4d')
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.network.neutron [req-8c1f7106-37b9-46d3-bc78-37014ea1c0a6 req-8349ec48-3f1c-45d6-8c69-5576d270c4b9 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Updated VIF entry in instance network info cache for port 540b7307-8fea-4b50-bab8-626983135a96. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.network.neutron [req-8c1f7106-37b9-46d3-bc78-37014ea1c0a6 req-8349ec48-3f1c-45d6-8c69-5576d270c4b9 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Updating instance_info_cache with network_info: [{"id": "540b7307-8fea-4b50-bab8-626983135a96", "address": "fa:16:3e:8d:a9:ab", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap540b7307-8f", "ovs_interfaceid": "540b7307-8fea-4b50-bab8-626983135a96", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8c1f7106-37b9-46d3-bc78-37014ea1c0a6 req-8349ec48-3f1c-45d6-8c69-5576d270c4b9 service nova] Releasing lock "refresh_cache-b23d4bcd-989a-45e1-89a3-6e3d30229a32" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] No VIF found with MAC fa:16:3e:ac:8c:49, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-c3a77492-e32a-4d27-a528-3c115f29ccbd req-7ccc0352-0c6b-471b-9a86-e762ef703774 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received event network-changed-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-c3a77492-e32a-4d27-a528-3c115f29ccbd req-7ccc0352-0c6b-471b-9a86-e762ef703774 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Refreshing instance network info cache due to event network-changed-cff588d4-f46a-4d44-a3e6-1c04f6abd380. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:30:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c3a77492-e32a-4d27-a528-3c115f29ccbd req-7ccc0352-0c6b-471b-9a86-e762ef703774 service nova] Acquiring lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270227.0138826, 2d0904c3-74e0-47bb-a832-758a7035b2f5 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:30:27 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] VM Resumed (Lifecycle Event)
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:30:27 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Instance spawned successfully.
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:30:27 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270227.0170276, 2d0904c3-74e0-47bb-a832-758a7035b2f5 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:30:27 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] VM Started (Lifecycle Event)
Apr 12 03:30:27 user nova-compute[72081]: INFO nova.compute.manager [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Took 15.00 seconds to spawn the instance on the hypervisor.
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.network.neutron [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Updating instance_info_cache with network_info: [{"id": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "address": "fa:16:3e:c0:f6:ee", "network": {"id": "b4464ad0-6022-43e7-bfe4-229e6a3a0600", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1704232678-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a027d7708e6747fe970c388d6ba66359", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcff588d4-f4", "ovs_interfaceid": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:27 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:30:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Releasing lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Instance network_info: |[{"id": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "address": "fa:16:3e:c0:f6:ee", "network": {"id": "b4464ad0-6022-43e7-bfe4-229e6a3a0600", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1704232678-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a027d7708e6747fe970c388d6ba66359", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcff588d4-f4", "ovs_interfaceid": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c3a77492-e32a-4d27-a528-3c115f29ccbd req-7ccc0352-0c6b-471b-9a86-e762ef703774 service nova] Acquired lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.network.neutron [req-c3a77492-e32a-4d27-a528-3c115f29ccbd req-7ccc0352-0c6b-471b-9a86-e762ef703774 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Refreshing network info cache for port cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Start _get_guest_xml network_info=[{"id": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "address": "fa:16:3e:c0:f6:ee", "network": {"id": "b4464ad0-6022-43e7-bfe4-229e6a3a0600", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1704232678-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a027d7708e6747fe970c388d6ba66359", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcff588d4-f4", "ovs_interfaceid": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:30:27 user nova-compute[72081]: INFO nova.compute.manager [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Took 16.69 seconds to build instance.
Apr 12 03:30:27 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:30:27 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:30:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerStableDeviceRescueTest-server-252473144',display_name='tempest-ServerStableDeviceRescueTest-server-252473144',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverstabledevicerescuetest-server-252473144',id=5,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBInWJADLpni9D9TfgnC1+/LMPrrQuD/lxuqixLR+mL8FYE3064UBREFALqAw20h2nHoOSsmv2cMRVuFj3Oi+UewdLxZXOi4pdekw/Cvt3tr0uQWDcvZgNGpzrP2V/qty5w==',key_name='tempest-keypair-31851495',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a027d7708e6747fe970c388d6ba66359',ramdisk_id='',reservation_id='r-30kyz70u',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerStableDeviceRescueTest-744520899',owner_user_name='tempest-ServerStableDeviceRescueTest-744520899-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:30:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='9405e57e55e84976b0caeba84e3aef62',uuid=72ba1218-065d-40c4-8503-7c4155e47503,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "address": "fa:16:3e:c0:f6:ee", "network": {"id": "b4464ad0-6022-43e7-bfe4-229e6a3a0600", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1704232678-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a027d7708e6747fe970c388d6ba66359", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcff588d4-f4", "ovs_interfaceid": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Converting VIF {"id": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "address": "fa:16:3e:c0:f6:ee", "network": {"id": "b4464ad0-6022-43e7-bfe4-229e6a3a0600", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1704232678-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a027d7708e6747fe970c388d6ba66359", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcff588d4-f4", "ovs_interfaceid": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:c0:f6:ee,bridge_name='br-int',has_traffic_filtering=True,id=cff588d4-f46a-4d44-a3e6-1c04f6abd380,network=Network(b4464ad0-6022-43e7-bfe4-229e6a3a0600),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapcff588d4-f4') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.objects.instance [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lazy-loading 'pci_devices' on Instance uuid 72ba1218-065d-40c4-8503-7c4155e47503 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-147a25b6-0562-44dc-96f7-722a986a50b7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "2d0904c3-74e0-47bb-a832-758a7035b2f5" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 17.018s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:30:27 user nova-compute[72081]:   <uuid>72ba1218-065d-40c4-8503-7c4155e47503</uuid>
Apr 12 03:30:27 user nova-compute[72081]:   <name>instance-00000005</name>
Apr 12 03:30:27 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:30:27 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:30:27 user nova-compute[72081]:   <metadata>
Apr 12 03:30:27 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:30:27 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:30:27 user nova-compute[72081]:       <nova:name>tempest-ServerStableDeviceRescueTest-server-252473144</nova:name>
Apr 12 03:30:27 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:30:27</nova:creationTime>
Apr 12 03:30:27 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:30:27 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:30:27 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:30:27 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:30:27 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:30:27 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:30:27 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:30:27 user nova-compute[72081]:       <nova:owner>
Apr 12 03:30:27 user nova-compute[72081]:         <nova:user uuid="9405e57e55e84976b0caeba84e3aef62">tempest-ServerStableDeviceRescueTest-744520899-project-member</nova:user>
Apr 12 03:30:27 user nova-compute[72081]:         <nova:project uuid="a027d7708e6747fe970c388d6ba66359">tempest-ServerStableDeviceRescueTest-744520899</nova:project>
Apr 12 03:30:27 user nova-compute[72081]:       </nova:owner>
Apr 12 03:30:27 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:30:27 user nova-compute[72081]:       <nova:ports>
Apr 12 03:30:27 user nova-compute[72081]:         <nova:port uuid="cff588d4-f46a-4d44-a3e6-1c04f6abd380">
Apr 12 03:30:27 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Apr 12 03:30:27 user nova-compute[72081]:         </nova:port>
Apr 12 03:30:27 user nova-compute[72081]:       </nova:ports>
Apr 12 03:30:27 user nova-compute[72081]:     </nova:instance>
Apr 12 03:30:27 user nova-compute[72081]:   </metadata>
Apr 12 03:30:27 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:30:27 user nova-compute[72081]:     <system>
Apr 12 03:30:27 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:30:27 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:30:27 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:30:27 user nova-compute[72081]:       <entry name="serial">72ba1218-065d-40c4-8503-7c4155e47503</entry>
Apr 12 03:30:27 user nova-compute[72081]:       <entry name="uuid">72ba1218-065d-40c4-8503-7c4155e47503</entry>
Apr 12 03:30:27 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:30:27 user nova-compute[72081]:     </system>
Apr 12 03:30:27 user nova-compute[72081]:   </sysinfo>
Apr 12 03:30:27 user nova-compute[72081]:   <os>
Apr 12 03:30:27 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:30:27 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:30:27 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:30:27 user nova-compute[72081]:   </os>
Apr 12 03:30:27 user nova-compute[72081]:   <features>
Apr 12 03:30:27 user nova-compute[72081]:     <acpi/>
Apr 12 03:30:27 user nova-compute[72081]:     <apic/>
Apr 12 03:30:27 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:30:27 user nova-compute[72081]:   </features>
Apr 12 03:30:27 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:30:27 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:30:27 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:30:27 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:30:27 user nova-compute[72081]:   </clock>
Apr 12 03:30:27 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:30:27 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:30:27 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:30:27 user nova-compute[72081]:   </cpu>
Apr 12 03:30:27 user nova-compute[72081]:   <devices>
Apr 12 03:30:27 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:30:27 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:30:27 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk"/>
Apr 12 03:30:27 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:30:27 user nova-compute[72081]:     </disk>
Apr 12 03:30:27 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:30:27 user nova-compute[72081]:       <mac address="fa:16:3e:c0:f6:ee"/>
Apr 12 03:30:27 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:30:27 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:30:27 user nova-compute[72081]:       <target dev="tapcff588d4-f4"/>
Apr 12 03:30:27 user nova-compute[72081]:     </interface>
Apr 12 03:30:27 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:30:27 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/console.log" append="off"/>
Apr 12 03:30:27 user nova-compute[72081]:     </serial>
Apr 12 03:30:27 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:30:27 user nova-compute[72081]:     <video>
Apr 12 03:30:27 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:30:27 user nova-compute[72081]:     </video>
Apr 12 03:30:27 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:30:27 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:30:27 user nova-compute[72081]:     </rng>
Apr 12 03:30:27 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:30:27 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:30:27 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:30:27 user nova-compute[72081]:     </memballoon>
Apr 12 03:30:27 user nova-compute[72081]:   </devices>
Apr 12 03:30:27 user nova-compute[72081]: </domain>
Apr 12 03:30:27 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:30:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerStableDeviceRescueTest-server-252473144',display_name='tempest-ServerStableDeviceRescueTest-server-252473144',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverstabledevicerescuetest-server-252473144',id=5,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBInWJADLpni9D9TfgnC1+/LMPrrQuD/lxuqixLR+mL8FYE3064UBREFALqAw20h2nHoOSsmv2cMRVuFj3Oi+UewdLxZXOi4pdekw/Cvt3tr0uQWDcvZgNGpzrP2V/qty5w==',key_name='tempest-keypair-31851495',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a027d7708e6747fe970c388d6ba66359',ramdisk_id='',reservation_id='r-30kyz70u',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerStableDeviceRescueTest-744520899',owner_user_name='tempest-ServerStableDeviceRescueTest-744520899-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:30:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='9405e57e55e84976b0caeba84e3aef62',uuid=72ba1218-065d-40c4-8503-7c4155e47503,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "address": "fa:16:3e:c0:f6:ee", "network": {"id": "b4464ad0-6022-43e7-bfe4-229e6a3a0600", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1704232678-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a027d7708e6747fe970c388d6ba66359", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcff588d4-f4", "ovs_interfaceid": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Converting VIF {"id": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "address": "fa:16:3e:c0:f6:ee", "network": {"id": "b4464ad0-6022-43e7-bfe4-229e6a3a0600", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1704232678-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a027d7708e6747fe970c388d6ba66359", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcff588d4-f4", "ovs_interfaceid": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:c0:f6:ee,bridge_name='br-int',has_traffic_filtering=True,id=cff588d4-f46a-4d44-a3e6-1c04f6abd380,network=Network(b4464ad0-6022-43e7-bfe4-229e6a3a0600),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapcff588d4-f4') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG os_vif [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:c0:f6:ee,bridge_name='br-int',has_traffic_filtering=True,id=cff588d4-f46a-4d44-a3e6-1c04f6abd380,network=Network(b4464ad0-6022-43e7-bfe4-229e6a3a0600),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapcff588d4-f4') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapcff588d4-f4, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapcff588d4-f4, col_values=(('external_ids', {'iface-id': 'cff588d4-f46a-4d44-a3e6-1c04f6abd380', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:c0:f6:ee', 'vm-uuid': '72ba1218-065d-40c4-8503-7c4155e47503'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270227.728442, 56834002-7593-4592-a86d-23bb90d00be0 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:30:27 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 56834002-7593-4592-a86d-23bb90d00be0] VM Resumed (Lifecycle Event)
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:27 user nova-compute[72081]: INFO os_vif [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:c0:f6:ee,bridge_name='br-int',has_traffic_filtering=True,id=cff588d4-f46a-4d44-a3e6-1c04f6abd380,network=Network(b4464ad0-6022-43e7-bfe4-229e6a3a0600),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapcff588d4-f4')
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.network.neutron [req-4a2e4b53-2ab1-4294-89d9-a5f7970f130c req-bd0d1c92-0410-467f-904f-f13a38b80d29 service nova] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Updated VIF entry in instance network info cache for port cc0a860c-4dc1-4a09-836a-3878bcb30d40. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.network.neutron [req-4a2e4b53-2ab1-4294-89d9-a5f7970f130c req-bd0d1c92-0410-467f-904f-f13a38b80d29 service nova] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Updating instance_info_cache with network_info: [{"id": "cc0a860c-4dc1-4a09-836a-3878bcb30d40", "address": "fa:16:3e:ac:8c:49", "network": {"id": "0f8e4d8d-8d90-47de-a965-5610331236ef", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-2097572913-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5dadbf1b0a124d2d929c85bc2b916022", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcc0a860c-4d", "ovs_interfaceid": "cc0a860c-4dc1-4a09-836a-3878bcb30d40", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:30:27 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Instance spawned successfully.
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4a2e4b53-2ab1-4294-89d9-a5f7970f130c req-bd0d1c92-0410-467f-904f-f13a38b80d29 service nova] Releasing lock "refresh_cache-ef4ab4cb-6164-47ea-be1d-cbe0698f991d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] No VIF found with MAC fa:16:3e:c0:f6:ee, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:27 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 56834002-7593-4592-a86d-23bb90d00be0] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:30:27 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270227.7317998, 56834002-7593-4592-a86d-23bb90d00be0 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:30:27 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 56834002-7593-4592-a86d-23bb90d00be0] VM Started (Lifecycle Event)
Apr 12 03:30:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:28 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:30:28 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:30:28 user nova-compute[72081]: INFO nova.compute.manager [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Took 16.74 seconds to spawn the instance on the hypervisor.
Apr 12 03:30:28 user nova-compute[72081]: DEBUG nova.compute.manager [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:30:28 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 56834002-7593-4592-a86d-23bb90d00be0] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:30:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-8c55728e-67d6-43ea-b124-8b7087fe7a32 req-ecc91852-c79e-4b44-9bc9-0004d89071e9 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Received event network-vif-plugged-540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:30:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8c55728e-67d6-43ea-b124-8b7087fe7a32 req-ecc91852-c79e-4b44-9bc9-0004d89071e9 service nova] Acquiring lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8c55728e-67d6-43ea-b124-8b7087fe7a32 req-ecc91852-c79e-4b44-9bc9-0004d89071e9 service nova] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8c55728e-67d6-43ea-b124-8b7087fe7a32 req-ecc91852-c79e-4b44-9bc9-0004d89071e9 service nova] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-8c55728e-67d6-43ea-b124-8b7087fe7a32 req-ecc91852-c79e-4b44-9bc9-0004d89071e9 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] No waiting events found dispatching network-vif-plugged-540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:30:28 user nova-compute[72081]: WARNING nova.compute.manager [req-8c55728e-67d6-43ea-b124-8b7087fe7a32 req-ecc91852-c79e-4b44-9bc9-0004d89071e9 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Received unexpected event network-vif-plugged-540b7307-8fea-4b50-bab8-626983135a96 for instance with vm_state building and task_state spawning.
Apr 12 03:30:28 user nova-compute[72081]: INFO nova.compute.manager [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Took 17.87 seconds to build instance.
Apr 12 03:30:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-319c1db8-7f15-4946-915d-e5b93087ab2c tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "56834002-7593-4592-a86d-23bb90d00be0" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 18.115s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:28 user nova-compute[72081]: DEBUG nova.network.neutron [req-c3a77492-e32a-4d27-a528-3c115f29ccbd req-7ccc0352-0c6b-471b-9a86-e762ef703774 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Updated VIF entry in instance network info cache for port cff588d4-f46a-4d44-a3e6-1c04f6abd380. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:30:28 user nova-compute[72081]: DEBUG nova.network.neutron [req-c3a77492-e32a-4d27-a528-3c115f29ccbd req-7ccc0352-0c6b-471b-9a86-e762ef703774 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Updating instance_info_cache with network_info: [{"id": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "address": "fa:16:3e:c0:f6:ee", "network": {"id": "b4464ad0-6022-43e7-bfe4-229e6a3a0600", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1704232678-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a027d7708e6747fe970c388d6ba66359", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcff588d4-f4", "ovs_interfaceid": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:30:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c3a77492-e32a-4d27-a528-3c115f29ccbd req-7ccc0352-0c6b-471b-9a86-e762ef703774 service nova] Releasing lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:30:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270230.1453638, b23d4bcd-989a-45e1-89a3-6e3d30229a32 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:30:30 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] VM Resumed (Lifecycle Event)
Apr 12 03:30:30 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:30:30 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Instance spawned successfully.
Apr 12 03:30:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:30 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:30:30 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270230.1509893, b23d4bcd-989a-45e1-89a3-6e3d30229a32 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:30:30 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] VM Started (Lifecycle Event)
Apr 12 03:30:30 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:30:30 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:30:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-00831d95-64a8-4927-8e74-1c9ad6fa2348 req-9d0d6729-0348-4db2-ae4c-d8a2f0c74179 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Received event network-vif-plugged-540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-00831d95-64a8-4927-8e74-1c9ad6fa2348 req-9d0d6729-0348-4db2-ae4c-d8a2f0c74179 service nova] Acquiring lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-00831d95-64a8-4927-8e74-1c9ad6fa2348 req-9d0d6729-0348-4db2-ae4c-d8a2f0c74179 service nova] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-00831d95-64a8-4927-8e74-1c9ad6fa2348 req-9d0d6729-0348-4db2-ae4c-d8a2f0c74179 service nova] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-00831d95-64a8-4927-8e74-1c9ad6fa2348 req-9d0d6729-0348-4db2-ae4c-d8a2f0c74179 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] No waiting events found dispatching network-vif-plugged-540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:30:30 user nova-compute[72081]: WARNING nova.compute.manager [req-00831d95-64a8-4927-8e74-1c9ad6fa2348 req-9d0d6729-0348-4db2-ae4c-d8a2f0c74179 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Received unexpected event network-vif-plugged-540b7307-8fea-4b50-bab8-626983135a96 for instance with vm_state building and task_state spawning.
Apr 12 03:30:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-00831d95-64a8-4927-8e74-1c9ad6fa2348 req-9d0d6729-0348-4db2-ae4c-d8a2f0c74179 service nova] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Received event network-vif-plugged-cc0a860c-4dc1-4a09-836a-3878bcb30d40 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-00831d95-64a8-4927-8e74-1c9ad6fa2348 req-9d0d6729-0348-4db2-ae4c-d8a2f0c74179 service nova] Acquiring lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-00831d95-64a8-4927-8e74-1c9ad6fa2348 req-9d0d6729-0348-4db2-ae4c-d8a2f0c74179 service nova] Lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-00831d95-64a8-4927-8e74-1c9ad6fa2348 req-9d0d6729-0348-4db2-ae4c-d8a2f0c74179 service nova] Lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-00831d95-64a8-4927-8e74-1c9ad6fa2348 req-9d0d6729-0348-4db2-ae4c-d8a2f0c74179 service nova] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] No waiting events found dispatching network-vif-plugged-cc0a860c-4dc1-4a09-836a-3878bcb30d40 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:30:30 user nova-compute[72081]: WARNING nova.compute.manager [req-00831d95-64a8-4927-8e74-1c9ad6fa2348 req-9d0d6729-0348-4db2-ae4c-d8a2f0c74179 service nova] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Received unexpected event network-vif-plugged-cc0a860c-4dc1-4a09-836a-3878bcb30d40 for instance with vm_state building and task_state spawning.
Apr 12 03:30:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-00831d95-64a8-4927-8e74-1c9ad6fa2348 req-9d0d6729-0348-4db2-ae4c-d8a2f0c74179 service nova] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Received event network-vif-plugged-cc0a860c-4dc1-4a09-836a-3878bcb30d40 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-00831d95-64a8-4927-8e74-1c9ad6fa2348 req-9d0d6729-0348-4db2-ae4c-d8a2f0c74179 service nova] Acquiring lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-00831d95-64a8-4927-8e74-1c9ad6fa2348 req-9d0d6729-0348-4db2-ae4c-d8a2f0c74179 service nova] Lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-00831d95-64a8-4927-8e74-1c9ad6fa2348 req-9d0d6729-0348-4db2-ae4c-d8a2f0c74179 service nova] Lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-00831d95-64a8-4927-8e74-1c9ad6fa2348 req-9d0d6729-0348-4db2-ae4c-d8a2f0c74179 service nova] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] No waiting events found dispatching network-vif-plugged-cc0a860c-4dc1-4a09-836a-3878bcb30d40 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:30:30 user nova-compute[72081]: WARNING nova.compute.manager [req-00831d95-64a8-4927-8e74-1c9ad6fa2348 req-9d0d6729-0348-4db2-ae4c-d8a2f0c74179 service nova] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Received unexpected event network-vif-plugged-cc0a860c-4dc1-4a09-836a-3878bcb30d40 for instance with vm_state building and task_state spawning.
Apr 12 03:30:30 user nova-compute[72081]: INFO nova.compute.manager [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Took 17.82 seconds to spawn the instance on the hypervisor.
Apr 12 03:30:30 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:30:30 user nova-compute[72081]: INFO nova.compute.manager [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Took 19.31 seconds to build instance.
Apr 12 03:30:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0cacaa5c-1afc-4a2a-aa98-992205d778f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 19.551s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-b71b81e1-dda9-405f-aa9a-579c9a356d67 req-277663ce-3c96-420f-904a-31167962302b service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received event network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b71b81e1-dda9-405f-aa9a-579c9a356d67 req-277663ce-3c96-420f-904a-31167962302b service nova] Acquiring lock "72ba1218-065d-40c4-8503-7c4155e47503-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b71b81e1-dda9-405f-aa9a-579c9a356d67 req-277663ce-3c96-420f-904a-31167962302b service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b71b81e1-dda9-405f-aa9a-579c9a356d67 req-277663ce-3c96-420f-904a-31167962302b service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-b71b81e1-dda9-405f-aa9a-579c9a356d67 req-277663ce-3c96-420f-904a-31167962302b service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] No waiting events found dispatching network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:30:30 user nova-compute[72081]: WARNING nova.compute.manager [req-b71b81e1-dda9-405f-aa9a-579c9a356d67 req-277663ce-3c96-420f-904a-31167962302b service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received unexpected event network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 for instance with vm_state building and task_state spawning.
Apr 12 03:30:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270232.5421786, 72ba1218-065d-40c4-8503-7c4155e47503 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:30:32 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] VM Resumed (Lifecycle Event)
Apr 12 03:30:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.compute.manager [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.compute.manager [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:30:32 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Instance spawned successfully.
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:30:32 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Instance spawned successfully.
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:30:32 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270232.5499094, 72ba1218-065d-40c4-8503-7c4155e47503 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:30:32 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] VM Started (Lifecycle Event)
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Acquiring lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:30:32 user nova-compute[72081]: INFO nova.compute.manager [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Took 18.04 seconds to spawn the instance on the hypervisor.
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.compute.manager [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:30:32 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270232.5941534, ef4ab4cb-6164-47ea-be1d-cbe0698f991d => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:30:32 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] VM Resumed (Lifecycle Event)
Apr 12 03:30:32 user nova-compute[72081]: DEBUG nova.compute.manager [req-df41b090-db7d-4f7f-892a-1a5db647e49b req-fc00a419-1059-4a7b-97a1-de01b4d689fd service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received event network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:30:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-df41b090-db7d-4f7f-892a-1a5db647e49b req-fc00a419-1059-4a7b-97a1-de01b4d689fd service nova] Acquiring lock "72ba1218-065d-40c4-8503-7c4155e47503-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-df41b090-db7d-4f7f-892a-1a5db647e49b req-fc00a419-1059-4a7b-97a1-de01b4d689fd service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.003s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-df41b090-db7d-4f7f-892a-1a5db647e49b req-fc00a419-1059-4a7b-97a1-de01b4d689fd service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-df41b090-db7d-4f7f-892a-1a5db647e49b req-fc00a419-1059-4a7b-97a1-de01b4d689fd service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] No waiting events found dispatching network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:30:33 user nova-compute[72081]: WARNING nova.compute.manager [req-df41b090-db7d-4f7f-892a-1a5db647e49b req-fc00a419-1059-4a7b-97a1-de01b4d689fd service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received unexpected event network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 for instance with vm_state building and task_state spawning.
Apr 12 03:30:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:30:33 user nova-compute[72081]: INFO nova.compute.manager [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Took 16.49 seconds to spawn the instance on the hypervisor.
Apr 12 03:30:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:30:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:33 user nova-compute[72081]: INFO nova.compute.manager [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Took 19.18 seconds to build instance.
Apr 12 03:30:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:30:33 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:30:33 user nova-compute[72081]: INFO nova.compute.claims [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Claim successful on node user
Apr 12 03:30:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-14b21ce6-cc29-420a-bf78-0b1b8db41ecd tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 19.312s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:33 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270232.5978496, ef4ab4cb-6164-47ea-be1d-cbe0698f991d => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:30:33 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] VM Started (Lifecycle Event)
Apr 12 03:30:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:30:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:30:33 user nova-compute[72081]: INFO nova.compute.manager [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Took 17.60 seconds to build instance.
Apr 12 03:30:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2210a820-7536-4aef-aa56-42e3c1adef9f tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "72ba1218-065d-40c4-8503-7c4155e47503" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 17.747s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:33 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:30:33 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:30:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.742s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:30:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:30:33 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:30:33 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:30:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:30:34 user nova-compute[72081]: DEBUG nova.policy [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '09fd9f0996bd48628564e6c170c11a17', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f51faa215a1546fbaa0db25c9773ba6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:30:34 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:30:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:30:34 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Creating image(s)
Apr 12 03:30:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Acquiring lock "/opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "/opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "/opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.194s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.007s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.206s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7/disk 1073741824" returned: 0 in 0.068s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.281s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.206s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:34 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Checking if we can resize image /opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:30:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7/disk --force-share --output=json" returned: 0 in 0.255s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:35 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Cannot resize image /opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:30:35 user nova-compute[72081]: DEBUG nova.objects.instance [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lazy-loading 'migration_context' on Instance uuid ccf9f424-d230-4079-87c5-bb08bc85b0b7 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:30:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:30:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Ensure instance console log exists: /opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:30:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:35 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Successfully created port: b58865c9-a096-40af-91cf-828e5131ea22 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:30:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:36 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Successfully updated port: b58865c9-a096-40af-91cf-828e5131ea22 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:30:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Acquiring lock "refresh_cache-ccf9f424-d230-4079-87c5-bb08bc85b0b7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:30:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Acquired lock "refresh_cache-ccf9f424-d230-4079-87c5-bb08bc85b0b7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:30:36 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:30:36 user nova-compute[72081]: DEBUG nova.compute.manager [req-e5ab2d91-27cb-4d73-8f46-08013891cc41 req-fb3fbe28-3e8a-4bc1-895d-c63f0eaaa361 service nova] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Received event network-changed-b58865c9-a096-40af-91cf-828e5131ea22 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:30:36 user nova-compute[72081]: DEBUG nova.compute.manager [req-e5ab2d91-27cb-4d73-8f46-08013891cc41 req-fb3fbe28-3e8a-4bc1-895d-c63f0eaaa361 service nova] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Refreshing instance network info cache due to event network-changed-b58865c9-a096-40af-91cf-828e5131ea22. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:30:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e5ab2d91-27cb-4d73-8f46-08013891cc41 req-fb3fbe28-3e8a-4bc1-895d-c63f0eaaa361 service nova] Acquiring lock "refresh_cache-ccf9f424-d230-4079-87c5-bb08bc85b0b7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:30:36 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Updating instance_info_cache with network_info: [{"id": "b58865c9-a096-40af-91cf-828e5131ea22", "address": "fa:16:3e:73:fe:b6", "network": {"id": "554fd203-5ff6-4282-87b1-d27aa5dba9e3", "bridge": "br-int", "label": "tempest-TestVolumeSwap-218878042-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f51faa215a1546fbaa0db25c9773ba6f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb58865c9-a0", "ovs_interfaceid": "b58865c9-a096-40af-91cf-828e5131ea22", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Releasing lock "refresh_cache-ccf9f424-d230-4079-87c5-bb08bc85b0b7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Instance network_info: |[{"id": "b58865c9-a096-40af-91cf-828e5131ea22", "address": "fa:16:3e:73:fe:b6", "network": {"id": "554fd203-5ff6-4282-87b1-d27aa5dba9e3", "bridge": "br-int", "label": "tempest-TestVolumeSwap-218878042-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f51faa215a1546fbaa0db25c9773ba6f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb58865c9-a0", "ovs_interfaceid": "b58865c9-a096-40af-91cf-828e5131ea22", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e5ab2d91-27cb-4d73-8f46-08013891cc41 req-fb3fbe28-3e8a-4bc1-895d-c63f0eaaa361 service nova] Acquired lock "refresh_cache-ccf9f424-d230-4079-87c5-bb08bc85b0b7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.network.neutron [req-e5ab2d91-27cb-4d73-8f46-08013891cc41 req-fb3fbe28-3e8a-4bc1-895d-c63f0eaaa361 service nova] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Refreshing network info cache for port b58865c9-a096-40af-91cf-828e5131ea22 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Start _get_guest_xml network_info=[{"id": "b58865c9-a096-40af-91cf-828e5131ea22", "address": "fa:16:3e:73:fe:b6", "network": {"id": "554fd203-5ff6-4282-87b1-d27aa5dba9e3", "bridge": "br-int", "label": "tempest-TestVolumeSwap-218878042-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f51faa215a1546fbaa0db25c9773ba6f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb58865c9-a0", "ovs_interfaceid": "b58865c9-a096-40af-91cf-828e5131ea22", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:30:37 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:30:37 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:30:31Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeSwap-server-571161248',display_name='tempest-TestVolumeSwap-server-571161248',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumeswap-server-571161248',id=6,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBL30RSbxjhGUn3xzLsIU4EV60BoTDcVspvaYQIbHfZgYoaBW0uNt/hjufJbiOeYVn8jEh3f2sIF4VJXBBn5+nGUjfajRfTCakLkfoH9ag8vOMyCnoG9vAKKOAoeQq5klzw==',key_name='tempest-keypair-586254963',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f51faa215a1546fbaa0db25c9773ba6f',ramdisk_id='',reservation_id='r-6re5vdrm',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeSwap-295510619',owner_user_name='tempest-TestVolumeSwap-295510619-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:30:34Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='09fd9f0996bd48628564e6c170c11a17',uuid=ccf9f424-d230-4079-87c5-bb08bc85b0b7,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "b58865c9-a096-40af-91cf-828e5131ea22", "address": "fa:16:3e:73:fe:b6", "network": {"id": "554fd203-5ff6-4282-87b1-d27aa5dba9e3", "bridge": "br-int", "label": "tempest-TestVolumeSwap-218878042-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f51faa215a1546fbaa0db25c9773ba6f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb58865c9-a0", "ovs_interfaceid": "b58865c9-a096-40af-91cf-828e5131ea22", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Converting VIF {"id": "b58865c9-a096-40af-91cf-828e5131ea22", "address": "fa:16:3e:73:fe:b6", "network": {"id": "554fd203-5ff6-4282-87b1-d27aa5dba9e3", "bridge": "br-int", "label": "tempest-TestVolumeSwap-218878042-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f51faa215a1546fbaa0db25c9773ba6f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb58865c9-a0", "ovs_interfaceid": "b58865c9-a096-40af-91cf-828e5131ea22", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:73:fe:b6,bridge_name='br-int',has_traffic_filtering=True,id=b58865c9-a096-40af-91cf-828e5131ea22,network=Network(554fd203-5ff6-4282-87b1-d27aa5dba9e3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb58865c9-a0') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.objects.instance [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lazy-loading 'pci_devices' on Instance uuid ccf9f424-d230-4079-87c5-bb08bc85b0b7 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:30:37 user nova-compute[72081]:   <uuid>ccf9f424-d230-4079-87c5-bb08bc85b0b7</uuid>
Apr 12 03:30:37 user nova-compute[72081]:   <name>instance-00000006</name>
Apr 12 03:30:37 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:30:37 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:30:37 user nova-compute[72081]:   <metadata>
Apr 12 03:30:37 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:30:37 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:30:37 user nova-compute[72081]:       <nova:name>tempest-TestVolumeSwap-server-571161248</nova:name>
Apr 12 03:30:37 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:30:37</nova:creationTime>
Apr 12 03:30:37 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:30:37 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:30:37 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:30:37 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:30:37 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:30:37 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:30:37 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:30:37 user nova-compute[72081]:       <nova:owner>
Apr 12 03:30:37 user nova-compute[72081]:         <nova:user uuid="09fd9f0996bd48628564e6c170c11a17">tempest-TestVolumeSwap-295510619-project-member</nova:user>
Apr 12 03:30:37 user nova-compute[72081]:         <nova:project uuid="f51faa215a1546fbaa0db25c9773ba6f">tempest-TestVolumeSwap-295510619</nova:project>
Apr 12 03:30:37 user nova-compute[72081]:       </nova:owner>
Apr 12 03:30:37 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:30:37 user nova-compute[72081]:       <nova:ports>
Apr 12 03:30:37 user nova-compute[72081]:         <nova:port uuid="b58865c9-a096-40af-91cf-828e5131ea22">
Apr 12 03:30:37 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Apr 12 03:30:37 user nova-compute[72081]:         </nova:port>
Apr 12 03:30:37 user nova-compute[72081]:       </nova:ports>
Apr 12 03:30:37 user nova-compute[72081]:     </nova:instance>
Apr 12 03:30:37 user nova-compute[72081]:   </metadata>
Apr 12 03:30:37 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:30:37 user nova-compute[72081]:     <system>
Apr 12 03:30:37 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:30:37 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:30:37 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:30:37 user nova-compute[72081]:       <entry name="serial">ccf9f424-d230-4079-87c5-bb08bc85b0b7</entry>
Apr 12 03:30:37 user nova-compute[72081]:       <entry name="uuid">ccf9f424-d230-4079-87c5-bb08bc85b0b7</entry>
Apr 12 03:30:37 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:30:37 user nova-compute[72081]:     </system>
Apr 12 03:30:37 user nova-compute[72081]:   </sysinfo>
Apr 12 03:30:37 user nova-compute[72081]:   <os>
Apr 12 03:30:37 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:30:37 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:30:37 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:30:37 user nova-compute[72081]:   </os>
Apr 12 03:30:37 user nova-compute[72081]:   <features>
Apr 12 03:30:37 user nova-compute[72081]:     <acpi/>
Apr 12 03:30:37 user nova-compute[72081]:     <apic/>
Apr 12 03:30:37 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:30:37 user nova-compute[72081]:   </features>
Apr 12 03:30:37 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:30:37 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:30:37 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:30:37 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:30:37 user nova-compute[72081]:   </clock>
Apr 12 03:30:37 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:30:37 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:30:37 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:30:37 user nova-compute[72081]:   </cpu>
Apr 12 03:30:37 user nova-compute[72081]:   <devices>
Apr 12 03:30:37 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:30:37 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:30:37 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7/disk"/>
Apr 12 03:30:37 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:30:37 user nova-compute[72081]:     </disk>
Apr 12 03:30:37 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:30:37 user nova-compute[72081]:       <mac address="fa:16:3e:73:fe:b6"/>
Apr 12 03:30:37 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:30:37 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:30:37 user nova-compute[72081]:       <target dev="tapb58865c9-a0"/>
Apr 12 03:30:37 user nova-compute[72081]:     </interface>
Apr 12 03:30:37 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:30:37 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7/console.log" append="off"/>
Apr 12 03:30:37 user nova-compute[72081]:     </serial>
Apr 12 03:30:37 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:30:37 user nova-compute[72081]:     <video>
Apr 12 03:30:37 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:30:37 user nova-compute[72081]:     </video>
Apr 12 03:30:37 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:30:37 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:30:37 user nova-compute[72081]:     </rng>
Apr 12 03:30:37 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:30:37 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:30:37 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:30:37 user nova-compute[72081]:     </memballoon>
Apr 12 03:30:37 user nova-compute[72081]:   </devices>
Apr 12 03:30:37 user nova-compute[72081]: </domain>
Apr 12 03:30:37 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:30:31Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeSwap-server-571161248',display_name='tempest-TestVolumeSwap-server-571161248',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumeswap-server-571161248',id=6,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBL30RSbxjhGUn3xzLsIU4EV60BoTDcVspvaYQIbHfZgYoaBW0uNt/hjufJbiOeYVn8jEh3f2sIF4VJXBBn5+nGUjfajRfTCakLkfoH9ag8vOMyCnoG9vAKKOAoeQq5klzw==',key_name='tempest-keypair-586254963',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f51faa215a1546fbaa0db25c9773ba6f',ramdisk_id='',reservation_id='r-6re5vdrm',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeSwap-295510619',owner_user_name='tempest-TestVolumeSwap-295510619-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:30:34Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='09fd9f0996bd48628564e6c170c11a17',uuid=ccf9f424-d230-4079-87c5-bb08bc85b0b7,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "b58865c9-a096-40af-91cf-828e5131ea22", "address": "fa:16:3e:73:fe:b6", "network": {"id": "554fd203-5ff6-4282-87b1-d27aa5dba9e3", "bridge": "br-int", "label": "tempest-TestVolumeSwap-218878042-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f51faa215a1546fbaa0db25c9773ba6f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb58865c9-a0", "ovs_interfaceid": "b58865c9-a096-40af-91cf-828e5131ea22", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Converting VIF {"id": "b58865c9-a096-40af-91cf-828e5131ea22", "address": "fa:16:3e:73:fe:b6", "network": {"id": "554fd203-5ff6-4282-87b1-d27aa5dba9e3", "bridge": "br-int", "label": "tempest-TestVolumeSwap-218878042-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f51faa215a1546fbaa0db25c9773ba6f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb58865c9-a0", "ovs_interfaceid": "b58865c9-a096-40af-91cf-828e5131ea22", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:73:fe:b6,bridge_name='br-int',has_traffic_filtering=True,id=b58865c9-a096-40af-91cf-828e5131ea22,network=Network(554fd203-5ff6-4282-87b1-d27aa5dba9e3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb58865c9-a0') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG os_vif [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:73:fe:b6,bridge_name='br-int',has_traffic_filtering=True,id=b58865c9-a096-40af-91cf-828e5131ea22,network=Network(554fd203-5ff6-4282-87b1-d27aa5dba9e3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb58865c9-a0') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapb58865c9-a0, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapb58865c9-a0, col_values=(('external_ids', {'iface-id': 'b58865c9-a096-40af-91cf-828e5131ea22', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:73:fe:b6', 'vm-uuid': 'ccf9f424-d230-4079-87c5-bb08bc85b0b7'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:30:37 user nova-compute[72081]: INFO os_vif [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:73:fe:b6,bridge_name='br-int',has_traffic_filtering=True,id=b58865c9-a096-40af-91cf-828e5131ea22,network=Network(554fd203-5ff6-4282-87b1-d27aa5dba9e3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb58865c9-a0')
Apr 12 03:30:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Acquiring lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] No VIF found with MAC fa:16:3e:73:fe:b6, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:30:37 user nova-compute[72081]: INFO nova.compute.claims [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Claim successful on node user
Apr 12 03:30:38 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:30:38 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:30:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.579s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:30:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:30:38 user nova-compute[72081]: DEBUG nova.network.neutron [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:30:38 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Ignoring supplied device name: /dev/sda. Libvirt can't honour user-supplied dev names
Apr 12 03:30:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:30:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:30:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:30:38 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Creating image(s)
Apr 12 03:30:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Acquiring lock "/opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "/opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "/opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Acquiring lock "c24b9043f1dedaa2da7a42314522ae21f8755c66" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "c24b9043f1dedaa2da7a42314522ae21f8755c66" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:38 user nova-compute[72081]: DEBUG nova.policy [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '431c372c4da24fa692f43f2d16764755', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90a92643aee44b66864aa10c2cf6f048', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:30:38 user nova-compute[72081]: DEBUG nova.network.neutron [req-e5ab2d91-27cb-4d73-8f46-08013891cc41 req-fb3fbe28-3e8a-4bc1-895d-c63f0eaaa361 service nova] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Updated VIF entry in instance network info cache for port b58865c9-a096-40af-91cf-828e5131ea22. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:30:38 user nova-compute[72081]: DEBUG nova.network.neutron [req-e5ab2d91-27cb-4d73-8f46-08013891cc41 req-fb3fbe28-3e8a-4bc1-895d-c63f0eaaa361 service nova] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Updating instance_info_cache with network_info: [{"id": "b58865c9-a096-40af-91cf-828e5131ea22", "address": "fa:16:3e:73:fe:b6", "network": {"id": "554fd203-5ff6-4282-87b1-d27aa5dba9e3", "bridge": "br-int", "label": "tempest-TestVolumeSwap-218878042-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f51faa215a1546fbaa0db25c9773ba6f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb58865c9-a0", "ovs_interfaceid": "b58865c9-a096-40af-91cf-828e5131ea22", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:30:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e5ab2d91-27cb-4d73-8f46-08013891cc41 req-fb3fbe28-3e8a-4bc1-895d-c63f0eaaa361 service nova] Releasing lock "refresh_cache-ccf9f424-d230-4079-87c5-bb08bc85b0b7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:30:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/c24b9043f1dedaa2da7a42314522ae21f8755c66.part --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/c24b9043f1dedaa2da7a42314522ae21f8755c66.part --force-share --output=json" returned: 0 in 0.186s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:39 user nova-compute[72081]: DEBUG nova.virt.images [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] 6aba2abf-651e-4da7-9a27-c2c991cb5c61 was qcow2, converting to raw {{(pid=72081) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 12 03:30:39 user nova-compute[72081]: DEBUG nova.privsep.utils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72081) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 12 03:30:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/c24b9043f1dedaa2da7a42314522ae21f8755c66.part /opt/stack/data/nova/instances/_base/c24b9043f1dedaa2da7a42314522ae21f8755c66.converted {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/c24b9043f1dedaa2da7a42314522ae21f8755c66.part /opt/stack/data/nova/instances/_base/c24b9043f1dedaa2da7a42314522ae21f8755c66.converted" returned: 0 in 0.232s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/c24b9043f1dedaa2da7a42314522ae21f8755c66.converted --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/c24b9043f1dedaa2da7a42314522ae21f8755c66.converted --force-share --output=json" returned: 0 in 0.183s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "c24b9043f1dedaa2da7a42314522ae21f8755c66" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.138s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/c24b9043f1dedaa2da7a42314522ae21f8755c66 --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/c24b9043f1dedaa2da7a42314522ae21f8755c66 --force-share --output=json" returned: 0 in 0.200s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Acquiring lock "c24b9043f1dedaa2da7a42314522ae21f8755c66" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "c24b9043f1dedaa2da7a42314522ae21f8755c66" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.004s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/c24b9043f1dedaa2da7a42314522ae21f8755c66 --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/c24b9043f1dedaa2da7a42314522ae21f8755c66 --force-share --output=json" returned: 0 in 0.150s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/c24b9043f1dedaa2da7a42314522ae21f8755c66,backing_fmt=raw /opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/c24b9043f1dedaa2da7a42314522ae21f8755c66,backing_fmt=raw /opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0/disk 1073741824" returned: 0 in 0.106s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "c24b9043f1dedaa2da7a42314522ae21f8755c66" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.264s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/c24b9043f1dedaa2da7a42314522ae21f8755c66 --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/c24b9043f1dedaa2da7a42314522ae21f8755c66 --force-share --output=json" returned: 0 in 0.170s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Checking if we can resize image /opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG nova.compute.manager [req-ea68aaf2-0804-4b77-baee-73d5d4acd7fc req-da13d549-a89e-49bd-b56e-22eb06493534 service nova] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Received event network-vif-plugged-b58865c9-a096-40af-91cf-828e5131ea22 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ea68aaf2-0804-4b77-baee-73d5d4acd7fc req-da13d549-a89e-49bd-b56e-22eb06493534 service nova] Acquiring lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ea68aaf2-0804-4b77-baee-73d5d4acd7fc req-da13d549-a89e-49bd-b56e-22eb06493534 service nova] Lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.003s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ea68aaf2-0804-4b77-baee-73d5d4acd7fc req-da13d549-a89e-49bd-b56e-22eb06493534 service nova] Lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG nova.compute.manager [req-ea68aaf2-0804-4b77-baee-73d5d4acd7fc req-da13d549-a89e-49bd-b56e-22eb06493534 service nova] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] No waiting events found dispatching network-vif-plugged-b58865c9-a096-40af-91cf-828e5131ea22 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:30:40 user nova-compute[72081]: WARNING nova.compute.manager [req-ea68aaf2-0804-4b77-baee-73d5d4acd7fc req-da13d549-a89e-49bd-b56e-22eb06493534 service nova] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Received unexpected event network-vif-plugged-b58865c9-a096-40af-91cf-828e5131ea22 for instance with vm_state building and task_state spawning.
Apr 12 03:30:40 user nova-compute[72081]: DEBUG nova.network.neutron [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Successfully created port: 7b7ff2ca-c0a4-4414-9eed-62bd43be3613 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0/disk --force-share --output=json" returned: 0 in 0.218s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Cannot resize image /opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG nova.objects.instance [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lazy-loading 'migration_context' on Instance uuid 413850ca-8175-4ed6-a6cf-ff42230fa1e0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Ensure instance console log exists: /opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:41 user nova-compute[72081]: DEBUG nova.network.neutron [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Successfully updated port: 7b7ff2ca-c0a4-4414-9eed-62bd43be3613 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:30:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Acquiring lock "refresh_cache-413850ca-8175-4ed6-a6cf-ff42230fa1e0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:30:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Acquired lock "refresh_cache-413850ca-8175-4ed6-a6cf-ff42230fa1e0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:30:41 user nova-compute[72081]: DEBUG nova.network.neutron [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:30:41 user nova-compute[72081]: DEBUG nova.network.neutron [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:30:41 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270241.9684844, ccf9f424-d230-4079-87c5-bb08bc85b0b7 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:30:41 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] VM Resumed (Lifecycle Event)
Apr 12 03:30:41 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:30:41 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:30:41 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:30:42 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Instance spawned successfully.
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:30:42 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270241.9689677, ccf9f424-d230-4079-87c5-bb08bc85b0b7 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:30:42 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] VM Started (Lifecycle Event)
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:30:42 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:30:42 user nova-compute[72081]: INFO nova.compute.manager [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Took 8.04 seconds to spawn the instance on the hypervisor.
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.network.neutron [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Updating instance_info_cache with network_info: [{"id": "7b7ff2ca-c0a4-4414-9eed-62bd43be3613", "address": "fa:16:3e:d2:cd:e1", "network": {"id": "01db77fb-7cb3-4eb1-b9d4-9ce9a671b333", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-539071762-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "90a92643aee44b66864aa10c2cf6f048", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7b7ff2ca-c0", "ovs_interfaceid": "7b7ff2ca-c0a4-4414-9eed-62bd43be3613", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Releasing lock "refresh_cache-413850ca-8175-4ed6-a6cf-ff42230fa1e0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Instance network_info: |[{"id": "7b7ff2ca-c0a4-4414-9eed-62bd43be3613", "address": "fa:16:3e:d2:cd:e1", "network": {"id": "01db77fb-7cb3-4eb1-b9d4-9ce9a671b333", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-539071762-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "90a92643aee44b66864aa10c2cf6f048", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7b7ff2ca-c0", "ovs_interfaceid": "7b7ff2ca-c0a4-4414-9eed-62bd43be3613", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Start _get_guest_xml network_info=[{"id": "7b7ff2ca-c0a4-4414-9eed-62bd43be3613", "address": "fa:16:3e:d2:cd:e1", "network": {"id": "01db77fb-7cb3-4eb1-b9d4-9ce9a671b333", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-539071762-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "90a92643aee44b66864aa10c2cf6f048", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7b7ff2ca-c0", "ovs_interfaceid": "7b7ff2ca-c0a4-4414-9eed-62bd43be3613", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'scsi', 'cdrom_bus': 'scsi', 'mapping': {'root': {'bus': 'scsi', 'dev': 'sda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'scsi', 'dev': 'sda', 'type': 'disk', 'boot_index': '1'}, 'disk.config': {'bus': 'scsi', 'dev': 'sdb', 'type': 'cdrom'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:30:30Z,direct_url=<?>,disk_format='qcow2',id=6aba2abf-651e-4da7-9a27-c2c991cb5c61,min_disk=0,min_ram=0,name='',owner='ad580ad6d0c64912a39f2561e766de88',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:30:33Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/sda', 'image': [{'disk_bus': 'scsi', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/sda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': '6aba2abf-651e-4da7-9a27-c2c991cb5c61'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:30:42 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:30:42 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:30:30Z,direct_url=<?>,disk_format='qcow2',id=6aba2abf-651e-4da7-9a27-c2c991cb5c61,min_disk=0,min_ram=0,name='',owner='ad580ad6d0c64912a39f2561e766de88',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:30:33Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:30:36Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachSCSIVolumeTestJSON-server-1054497631',display_name='tempest-AttachSCSIVolumeTestJSON-server-1054497631',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachscsivolumetestjson-server-1054497631',id=7,image_ref='6aba2abf-651e-4da7-9a27-c2c991cb5c61',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEW7Yrpu5SOQ/b67vmyb/sqyksJOlf4MPK63MIfzlSnm+OH9BJ8Yy9IStENYg+9gKakI2VPnKTMAt5l9nxKRjD2o/A0rxASwuMLoI7j1zUAFxuHzukBR9RVxm++z5vy2Ww==',key_name='tempest-keypair-539207312',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='90a92643aee44b66864aa10c2cf6f048',ramdisk_id='',reservation_id='r-dosfn8tv',resources=None,root_device_name='/dev/sda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='6aba2abf-651e-4da7-9a27-c2c991cb5c61',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='scsi',image_hw_disk_bus='scsi',image_hw_machine_type='pc',image_hw_scsi_model='virtio-scsi',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-AttachSCSIVolumeTestJSON-95582294',owner_user_name='tempest-AttachSCSIVolumeTestJSON-95582294-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:30:39Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='431c372c4da24fa692f43f2d16764755',uuid=413850ca-8175-4ed6-a6cf-ff42230fa1e0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7b7ff2ca-c0a4-4414-9eed-62bd43be3613", "address": "fa:16:3e:d2:cd:e1", "network": {"id": "01db77fb-7cb3-4eb1-b9d4-9ce9a671b333", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-539071762-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "90a92643aee44b66864aa10c2cf6f048", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7b7ff2ca-c0", "ovs_interfaceid": "7b7ff2ca-c0a4-4414-9eed-62bd43be3613", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Converting VIF {"id": "7b7ff2ca-c0a4-4414-9eed-62bd43be3613", "address": "fa:16:3e:d2:cd:e1", "network": {"id": "01db77fb-7cb3-4eb1-b9d4-9ce9a671b333", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-539071762-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "90a92643aee44b66864aa10c2cf6f048", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7b7ff2ca-c0", "ovs_interfaceid": "7b7ff2ca-c0a4-4414-9eed-62bd43be3613", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d2:cd:e1,bridge_name='br-int',has_traffic_filtering=True,id=7b7ff2ca-c0a4-4414-9eed-62bd43be3613,network=Network(01db77fb-7cb3-4eb1-b9d4-9ce9a671b333),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7b7ff2ca-c0') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.objects.instance [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lazy-loading 'pci_devices' on Instance uuid 413850ca-8175-4ed6-a6cf-ff42230fa1e0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:42 user nova-compute[72081]: INFO nova.compute.manager [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Took 9.54 seconds to build instance.
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:30:42 user nova-compute[72081]:   <uuid>413850ca-8175-4ed6-a6cf-ff42230fa1e0</uuid>
Apr 12 03:30:42 user nova-compute[72081]:   <name>instance-00000007</name>
Apr 12 03:30:42 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:30:42 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:30:42 user nova-compute[72081]:   <metadata>
Apr 12 03:30:42 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:30:42 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:30:42 user nova-compute[72081]:       <nova:name>tempest-AttachSCSIVolumeTestJSON-server-1054497631</nova:name>
Apr 12 03:30:42 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:30:42</nova:creationTime>
Apr 12 03:30:42 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:30:42 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:30:42 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:30:42 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:30:42 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:30:42 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:30:42 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:30:42 user nova-compute[72081]:       <nova:owner>
Apr 12 03:30:42 user nova-compute[72081]:         <nova:user uuid="431c372c4da24fa692f43f2d16764755">tempest-AttachSCSIVolumeTestJSON-95582294-project-member</nova:user>
Apr 12 03:30:42 user nova-compute[72081]:         <nova:project uuid="90a92643aee44b66864aa10c2cf6f048">tempest-AttachSCSIVolumeTestJSON-95582294</nova:project>
Apr 12 03:30:42 user nova-compute[72081]:       </nova:owner>
Apr 12 03:30:42 user nova-compute[72081]:       <nova:root type="image" uuid="6aba2abf-651e-4da7-9a27-c2c991cb5c61"/>
Apr 12 03:30:42 user nova-compute[72081]:       <nova:ports>
Apr 12 03:30:42 user nova-compute[72081]:         <nova:port uuid="7b7ff2ca-c0a4-4414-9eed-62bd43be3613">
Apr 12 03:30:42 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 12 03:30:42 user nova-compute[72081]:         </nova:port>
Apr 12 03:30:42 user nova-compute[72081]:       </nova:ports>
Apr 12 03:30:42 user nova-compute[72081]:     </nova:instance>
Apr 12 03:30:42 user nova-compute[72081]:   </metadata>
Apr 12 03:30:42 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:30:42 user nova-compute[72081]:     <system>
Apr 12 03:30:42 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:30:42 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:30:42 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:30:42 user nova-compute[72081]:       <entry name="serial">413850ca-8175-4ed6-a6cf-ff42230fa1e0</entry>
Apr 12 03:30:42 user nova-compute[72081]:       <entry name="uuid">413850ca-8175-4ed6-a6cf-ff42230fa1e0</entry>
Apr 12 03:30:42 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:30:42 user nova-compute[72081]:     </system>
Apr 12 03:30:42 user nova-compute[72081]:   </sysinfo>
Apr 12 03:30:42 user nova-compute[72081]:   <os>
Apr 12 03:30:42 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:30:42 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:30:42 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:30:42 user nova-compute[72081]:   </os>
Apr 12 03:30:42 user nova-compute[72081]:   <features>
Apr 12 03:30:42 user nova-compute[72081]:     <acpi/>
Apr 12 03:30:42 user nova-compute[72081]:     <apic/>
Apr 12 03:30:42 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:30:42 user nova-compute[72081]:   </features>
Apr 12 03:30:42 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:30:42 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:30:42 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:30:42 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:30:42 user nova-compute[72081]:   </clock>
Apr 12 03:30:42 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:30:42 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:30:42 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:30:42 user nova-compute[72081]:   </cpu>
Apr 12 03:30:42 user nova-compute[72081]:   <devices>
Apr 12 03:30:42 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:30:42 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:30:42 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0/disk"/>
Apr 12 03:30:42 user nova-compute[72081]:       <target dev="sda" bus="scsi"/>
Apr 12 03:30:42 user nova-compute[72081]:       <address type="drive" controller="0" unit="0"/>
Apr 12 03:30:42 user nova-compute[72081]:     </disk>
Apr 12 03:30:42 user nova-compute[72081]:     <disk type="file" device="cdrom">
Apr 12 03:30:42 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none"/>
Apr 12 03:30:42 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0/disk.config"/>
Apr 12 03:30:42 user nova-compute[72081]:       <target dev="sdb" bus="scsi"/>
Apr 12 03:30:42 user nova-compute[72081]:       <address type="drive" controller="0" unit="1"/>
Apr 12 03:30:42 user nova-compute[72081]:     </disk>
Apr 12 03:30:42 user nova-compute[72081]:     <controller type="scsi" index="0" model="virtio-scsi"/>
Apr 12 03:30:42 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:30:42 user nova-compute[72081]:       <mac address="fa:16:3e:d2:cd:e1"/>
Apr 12 03:30:42 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:30:42 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:30:42 user nova-compute[72081]:       <target dev="tap7b7ff2ca-c0"/>
Apr 12 03:30:42 user nova-compute[72081]:     </interface>
Apr 12 03:30:42 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:30:42 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0/console.log" append="off"/>
Apr 12 03:30:42 user nova-compute[72081]:     </serial>
Apr 12 03:30:42 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:30:42 user nova-compute[72081]:     <video>
Apr 12 03:30:42 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:30:42 user nova-compute[72081]:     </video>
Apr 12 03:30:42 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:30:42 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:30:42 user nova-compute[72081]:     </rng>
Apr 12 03:30:42 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:30:42 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:30:42 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:30:42 user nova-compute[72081]:     </memballoon>
Apr 12 03:30:42 user nova-compute[72081]:   </devices>
Apr 12 03:30:42 user nova-compute[72081]: </domain>
Apr 12 03:30:42 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:30:36Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachSCSIVolumeTestJSON-server-1054497631',display_name='tempest-AttachSCSIVolumeTestJSON-server-1054497631',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachscsivolumetestjson-server-1054497631',id=7,image_ref='6aba2abf-651e-4da7-9a27-c2c991cb5c61',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEW7Yrpu5SOQ/b67vmyb/sqyksJOlf4MPK63MIfzlSnm+OH9BJ8Yy9IStENYg+9gKakI2VPnKTMAt5l9nxKRjD2o/A0rxASwuMLoI7j1zUAFxuHzukBR9RVxm++z5vy2Ww==',key_name='tempest-keypair-539207312',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='90a92643aee44b66864aa10c2cf6f048',ramdisk_id='',reservation_id='r-dosfn8tv',resources=None,root_device_name='/dev/sda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='6aba2abf-651e-4da7-9a27-c2c991cb5c61',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='scsi',image_hw_disk_bus='scsi',image_hw_machine_type='pc',image_hw_scsi_model='virtio-scsi',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-AttachSCSIVolumeTestJSON-95582294',owner_user_name='tempest-AttachSCSIVolumeTestJSON-95582294-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:30:39Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='431c372c4da24fa692f43f2d16764755',uuid=413850ca-8175-4ed6-a6cf-ff42230fa1e0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7b7ff2ca-c0a4-4414-9eed-62bd43be3613", "address": "fa:16:3e:d2:cd:e1", "network": {"id": "01db77fb-7cb3-4eb1-b9d4-9ce9a671b333", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-539071762-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "90a92643aee44b66864aa10c2cf6f048", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7b7ff2ca-c0", "ovs_interfaceid": "7b7ff2ca-c0a4-4414-9eed-62bd43be3613", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Converting VIF {"id": "7b7ff2ca-c0a4-4414-9eed-62bd43be3613", "address": "fa:16:3e:d2:cd:e1", "network": {"id": "01db77fb-7cb3-4eb1-b9d4-9ce9a671b333", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-539071762-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "90a92643aee44b66864aa10c2cf6f048", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7b7ff2ca-c0", "ovs_interfaceid": "7b7ff2ca-c0a4-4414-9eed-62bd43be3613", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d2:cd:e1,bridge_name='br-int',has_traffic_filtering=True,id=7b7ff2ca-c0a4-4414-9eed-62bd43be3613,network=Network(01db77fb-7cb3-4eb1-b9d4-9ce9a671b333),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7b7ff2ca-c0') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG os_vif [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:d2:cd:e1,bridge_name='br-int',has_traffic_filtering=True,id=7b7ff2ca-c0a4-4414-9eed-62bd43be3613,network=Network(01db77fb-7cb3-4eb1-b9d4-9ce9a671b333),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7b7ff2ca-c0') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap7b7ff2ca-c0, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap7b7ff2ca-c0, col_values=(('external_ids', {'iface-id': '7b7ff2ca-c0a4-4414-9eed-62bd43be3613', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:d2:cd:e1', 'vm-uuid': '413850ca-8175-4ed6-a6cf-ff42230fa1e0'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e07b2ee-3fc5-48da-810e-0e431e971682 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 9.774s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:42 user nova-compute[72081]: INFO os_vif [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:d2:cd:e1,bridge_name='br-int',has_traffic_filtering=True,id=7b7ff2ca-c0a4-4414-9eed-62bd43be3613,network=Network(01db77fb-7cb3-4eb1-b9d4-9ce9a671b333),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7b7ff2ca-c0')
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.compute.manager [req-53f47c4a-46d1-4145-8777-617e6fd8cc70 req-74e19525-f0b1-41aa-be96-b25e1a50e7f6 service nova] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Received event network-vif-plugged-b58865c9-a096-40af-91cf-828e5131ea22 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-53f47c4a-46d1-4145-8777-617e6fd8cc70 req-74e19525-f0b1-41aa-be96-b25e1a50e7f6 service nova] Acquiring lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-53f47c4a-46d1-4145-8777-617e6fd8cc70 req-74e19525-f0b1-41aa-be96-b25e1a50e7f6 service nova] Lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-53f47c4a-46d1-4145-8777-617e6fd8cc70 req-74e19525-f0b1-41aa-be96-b25e1a50e7f6 service nova] Lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.compute.manager [req-53f47c4a-46d1-4145-8777-617e6fd8cc70 req-74e19525-f0b1-41aa-be96-b25e1a50e7f6 service nova] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] No waiting events found dispatching network-vif-plugged-b58865c9-a096-40af-91cf-828e5131ea22 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:30:42 user nova-compute[72081]: WARNING nova.compute.manager [req-53f47c4a-46d1-4145-8777-617e6fd8cc70 req-74e19525-f0b1-41aa-be96-b25e1a50e7f6 service nova] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Received unexpected event network-vif-plugged-b58865c9-a096-40af-91cf-828e5131ea22 for instance with vm_state active and task_state None.
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.compute.manager [req-53f47c4a-46d1-4145-8777-617e6fd8cc70 req-74e19525-f0b1-41aa-be96-b25e1a50e7f6 service nova] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Received event network-changed-7b7ff2ca-c0a4-4414-9eed-62bd43be3613 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.compute.manager [req-53f47c4a-46d1-4145-8777-617e6fd8cc70 req-74e19525-f0b1-41aa-be96-b25e1a50e7f6 service nova] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Refreshing instance network info cache due to event network-changed-7b7ff2ca-c0a4-4414-9eed-62bd43be3613. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-53f47c4a-46d1-4145-8777-617e6fd8cc70 req-74e19525-f0b1-41aa-be96-b25e1a50e7f6 service nova] Acquiring lock "refresh_cache-413850ca-8175-4ed6-a6cf-ff42230fa1e0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-53f47c4a-46d1-4145-8777-617e6fd8cc70 req-74e19525-f0b1-41aa-be96-b25e1a50e7f6 service nova] Acquired lock "refresh_cache-413850ca-8175-4ed6-a6cf-ff42230fa1e0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.network.neutron [req-53f47c4a-46d1-4145-8777-617e6fd8cc70 req-74e19525-f0b1-41aa-be96-b25e1a50e7f6 service nova] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Refreshing network info cache for port 7b7ff2ca-c0a4-4414-9eed-62bd43be3613 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] No BDM found with device name sda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] No BDM found with device name sdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:30:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] No VIF found with MAC fa:16:3e:d2:cd:e1, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:30:42 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Using config drive
Apr 12 03:30:43 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Creating config drive at /opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0/disk.config
Apr 12 03:30:43 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Running cmd (subprocess): genisoimage -o /opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpuon12o31 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:43 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] CMD "genisoimage -o /opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpuon12o31" returned: 0 in 0.099s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:44 user nova-compute[72081]: DEBUG nova.network.neutron [req-53f47c4a-46d1-4145-8777-617e6fd8cc70 req-74e19525-f0b1-41aa-be96-b25e1a50e7f6 service nova] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Updated VIF entry in instance network info cache for port 7b7ff2ca-c0a4-4414-9eed-62bd43be3613. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:30:44 user nova-compute[72081]: DEBUG nova.network.neutron [req-53f47c4a-46d1-4145-8777-617e6fd8cc70 req-74e19525-f0b1-41aa-be96-b25e1a50e7f6 service nova] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Updating instance_info_cache with network_info: [{"id": "7b7ff2ca-c0a4-4414-9eed-62bd43be3613", "address": "fa:16:3e:d2:cd:e1", "network": {"id": "01db77fb-7cb3-4eb1-b9d4-9ce9a671b333", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-539071762-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "90a92643aee44b66864aa10c2cf6f048", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7b7ff2ca-c0", "ovs_interfaceid": "7b7ff2ca-c0a4-4414-9eed-62bd43be3613", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:30:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-53f47c4a-46d1-4145-8777-617e6fd8cc70 req-74e19525-f0b1-41aa-be96-b25e1a50e7f6 service nova] Releasing lock "refresh_cache-413850ca-8175-4ed6-a6cf-ff42230fa1e0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:30:45 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:45 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:45 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:45 user nova-compute[72081]: DEBUG nova.compute.manager [req-b781b542-5820-466c-b173-5ebe1cabc20b req-1bb605b7-47c8-4eb9-adb1-84ba9cbaee48 service nova] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Received event network-vif-plugged-7b7ff2ca-c0a4-4414-9eed-62bd43be3613 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:30:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b781b542-5820-466c-b173-5ebe1cabc20b req-1bb605b7-47c8-4eb9-adb1-84ba9cbaee48 service nova] Acquiring lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b781b542-5820-466c-b173-5ebe1cabc20b req-1bb605b7-47c8-4eb9-adb1-84ba9cbaee48 service nova] Lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b781b542-5820-466c-b173-5ebe1cabc20b req-1bb605b7-47c8-4eb9-adb1-84ba9cbaee48 service nova] Lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:45 user nova-compute[72081]: DEBUG nova.compute.manager [req-b781b542-5820-466c-b173-5ebe1cabc20b req-1bb605b7-47c8-4eb9-adb1-84ba9cbaee48 service nova] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] No waiting events found dispatching network-vif-plugged-7b7ff2ca-c0a4-4414-9eed-62bd43be3613 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:30:45 user nova-compute[72081]: WARNING nova.compute.manager [req-b781b542-5820-466c-b173-5ebe1cabc20b req-1bb605b7-47c8-4eb9-adb1-84ba9cbaee48 service nova] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Received unexpected event network-vif-plugged-7b7ff2ca-c0a4-4414-9eed-62bd43be3613 for instance with vm_state building and task_state spawning.
Apr 12 03:30:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:48 user nova-compute[72081]: DEBUG nova.compute.manager [req-03cfd944-93b7-4706-b02e-64560c370632 req-d2262ea9-cb8a-41cd-a353-5065a8ca96a0 service nova] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Received event network-vif-plugged-7b7ff2ca-c0a4-4414-9eed-62bd43be3613 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:30:48 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-03cfd944-93b7-4706-b02e-64560c370632 req-d2262ea9-cb8a-41cd-a353-5065a8ca96a0 service nova] Acquiring lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:48 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-03cfd944-93b7-4706-b02e-64560c370632 req-d2262ea9-cb8a-41cd-a353-5065a8ca96a0 service nova] Lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:48 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-03cfd944-93b7-4706-b02e-64560c370632 req-d2262ea9-cb8a-41cd-a353-5065a8ca96a0 service nova] Lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:48 user nova-compute[72081]: DEBUG nova.compute.manager [req-03cfd944-93b7-4706-b02e-64560c370632 req-d2262ea9-cb8a-41cd-a353-5065a8ca96a0 service nova] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] No waiting events found dispatching network-vif-plugged-7b7ff2ca-c0a4-4414-9eed-62bd43be3613 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:30:48 user nova-compute[72081]: WARNING nova.compute.manager [req-03cfd944-93b7-4706-b02e-64560c370632 req-d2262ea9-cb8a-41cd-a353-5065a8ca96a0 service nova] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Received unexpected event network-vif-plugged-7b7ff2ca-c0a4-4414-9eed-62bd43be3613 for instance with vm_state building and task_state spawning.
Apr 12 03:30:48 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:30:48 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:30:48 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270248.0971336, 413850ca-8175-4ed6-a6cf-ff42230fa1e0 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:30:48 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] VM Resumed (Lifecycle Event)
Apr 12 03:30:48 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Instance spawned successfully.
Apr 12 03:30:48 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Attempting to register defaults for the following image properties: ['hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:30:48 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:48 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:48 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:48 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:30:48 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:30:48 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:30:48 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:30:48 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270248.1060307, 413850ca-8175-4ed6-a6cf-ff42230fa1e0 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:30:48 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] VM Started (Lifecycle Event)
Apr 12 03:30:48 user nova-compute[72081]: INFO nova.compute.manager [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Took 9.54 seconds to spawn the instance on the hypervisor.
Apr 12 03:30:48 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:30:48 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:30:48 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:30:48 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:30:48 user nova-compute[72081]: INFO nova.compute.manager [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Took 10.74 seconds to build instance.
Apr 12 03:30:48 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f0d8c91-b183-4533-9dfa-b4ea54f1d540 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 10.911s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:52 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:30:52 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:30:52 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Cleaning up deleted instances {{(pid=72081) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 12 03:30:52 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] There are 0 instances to clean {{(pid=72081) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 12 03:30:52 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:30:52 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Cleaning up deleted instances with incomplete migration  {{(pid=72081) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 12 03:30:52 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:30:55 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:30:55 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:30:55 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:30:55 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:30:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:57 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:30:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:57 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:30:57 user nova-compute[72081]: INFO nova.compute.claims [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Claim successful on node user
Apr 12 03:30:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:30:57 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:30:57 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:30:57 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:30:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:57 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:30:57 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:30:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.490s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:57 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:30:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.271s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:57 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:58 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:30:58 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:30:58 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Creating image(s)
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "/opt/stack/data/nova/instances/cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "/opt/stack/data/nova/instances/cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "/opt/stack/data/nova/instances/cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.006s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:30:58 user nova-compute[72081]: INFO nova.compute.claims [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Claim successful on node user
Apr 12 03:30:58 user nova-compute[72081]: DEBUG nova.policy [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '84ea3a8776a944178ba5a04bce4d9b05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86f7b42b161f41efb234f0ce70210a08', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json" returned: 0 in 0.288s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.356s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json" returned: 0 in 0.231s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.161s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0/disk 1073741824" returned: 0 in 0.056s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.222s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Checking if we can resize image /opt/stack/data/nova/instances/cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:30:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Cannot resize image /opt/stack/data/nova/instances/cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG nova.objects.instance [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lazy-loading 'migration_context' on Instance uuid cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Ensure instance console log exists: /opt/stack/data/nova/instances/cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json" returned: 0 in 0.163s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 1.245s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:59 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:30:59 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ef4ab4cb-6164-47ea-be1d-cbe0698f991d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:59 user nova-compute[72081]: INFO nova.virt.block_device [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Booting with volume ba2eb790-6032-4455-82ed-af81c61ca352 at /dev/vda
Apr 12 03:30:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ef4ab4cb-6164-47ea-be1d-cbe0698f991d/disk --force-share --output=json" returned: 0 in 0.160s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ef4ab4cb-6164-47ea-be1d-cbe0698f991d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG os_brick.utils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:30:59 user nova-compute[72081]: INFO oslo.privsep.daemon [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Running privsep helper: ['sudo', 'nova-rootwrap', '/etc/nova/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/nova/nova-cpu.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmpp46zmy22/privsep.sock']
Apr 12 03:30:59 user sudo[82042]:    stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/nova-rootwrap /etc/nova/rootwrap.conf privsep-helper --config-file /etc/nova/nova-cpu.conf --privsep_context os_brick.privileged.default --privsep_sock_path /tmp/tmpp46zmy22/privsep.sock
Apr 12 03:30:59 user sudo[82042]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1001)
Apr 12 03:30:59 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Successfully created port: 91c0bcc3-17a9-4444-bb3c-715e45188260 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ef4ab4cb-6164-47ea-be1d-cbe0698f991d/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:30:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:00 user nova-compute[72081]: DEBUG nova.policy [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'e7aeea82653d4681aa22f6dd8ae742b4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72d31bafe182459aa8bcb4f1bbb5e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:31:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7/disk --force-share --output=json" returned: 0 in 0.162s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0/disk --force-share --output=json" returned: 0 in 0.182s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0/disk --force-share --output=json" returned: 0 in 0.174s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:01 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:31:01 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:31:01 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=7806MB free_disk=26.469470977783203GB free_vcpus=5 pci_devices=[{"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:01 user sudo[82042]: pam_unix(sudo:session): session closed for user root
Apr 12 03:31:01 user nova-compute[72081]: INFO oslo.privsep.daemon [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Spawned new privsep daemon via rootwrap
Apr 12 03:31:01 user nova-compute[72081]: INFO oslo.privsep.daemon [-] privsep daemon starting
Apr 12 03:31:01 user nova-compute[72081]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
Apr 12 03:31:01 user nova-compute[72081]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none
Apr 12 03:31:01 user nova-compute[72081]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 82083
Apr 12 03:31:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0feb7434-69ca-4581-88bf-c7888495a80d]: (2,) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 56834002-7593-4592-a86d-23bb90d00be0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 2d0904c3-74e0-47bb-a832-758a7035b2f5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance b23d4bcd-989a-45e1-89a3-6e3d30229a32 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance ef4ab4cb-6164-47ea-be1d-cbe0698f991d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 72ba1218-065d-40c4-8503-7c4155e47503 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance ccf9f424-d230-4079-87c5-bb08bc85b0b7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 413850ca-8175-4ed6-a6cf-ff42230fa1e0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 7119b8b3-291b-4868-820e-d7bd3dd3ada4 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=1664MB phys_disk=40GB used_disk=8GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Refreshing inventories for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Updating ProviderTree inventory for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Updating inventory in ProviderTree for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Refreshing aggregate associations for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6, aggregates: None {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Refreshing trait associations for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6, traits: COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_USB,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_RESCUE_BFV,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE41,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_STORAGE_BUS_FDC,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_STORAGE_BUS_IDE,COMPUTE_TRUSTED_CERTS,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_GRAPHICS_MODEL_CIRRUS,HW_CPU_X86_SSSE3,HW_CPU_X86_SSE,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_EXTEND,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_STORAGE_BUS_SATA,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_GRAPHICS_MODEL_QXL,HW_CPU_X86_SSE42,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_ACCELERATORS,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,HW_CPU_X86_MMX,COMPUTE_NODE,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_DEVICE_TAGGING,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_VIOMMU_MODEL_AUTO {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d531d094-0788-466a-bd76-9d2fb7810190]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:01 user nova-compute[72081]: WARNING os_brick.initiator.connectors.nvmeof [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Could not find nvme_core/parameters/multipath: FileNotFoundError: [Errno 2] No such file or directory: '/sys/module/nvme_core/parameters/multipath'
Apr 12 03:31:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7369c47c-365f-438f-84c9-ae1e0847cba0]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f92b4b8-8058-4636-ab80-1940283205cd]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f5bdf1ca-7569-4a95-9bba-c54315888f6c]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Successfully updated port: 91c0bcc3-17a9-4444-bb3c-715e45188260 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG nova.compute.manager [req-7ac4f5ea-23c0-4b50-b125-a98271eb666b req-0ebda4ba-8919-49cd-b01c-7c120a8b11a6 service nova] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Received event network-changed-91c0bcc3-17a9-4444-bb3c-715e45188260 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG nova.compute.manager [req-7ac4f5ea-23c0-4b50-b125-a98271eb666b req-0ebda4ba-8919-49cd-b01c-7c120a8b11a6 service nova] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Refreshing instance network info cache due to event network-changed-91c0bcc3-17a9-4444-bb3c-715e45188260. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7ac4f5ea-23c0-4b50-b125-a98271eb666b req-0ebda4ba-8919-49cd-b01c-7c120a8b11a6 service nova] Acquiring lock "refresh_cache-cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7ac4f5ea-23c0-4b50-b125-a98271eb666b req-0ebda4ba-8919-49cd-b01c-7c120a8b11a6 service nova] Acquired lock "refresh_cache-cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG nova.network.neutron [req-7ac4f5ea-23c0-4b50-b125-a98271eb666b req-0ebda4ba-8919-49cd-b01c-7c120a8b11a6 service nova] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Refreshing network info cache for port 91c0bcc3-17a9-4444-bb3c-715e45188260 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "refresh_cache-cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:31:01 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:31:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9791b505-aec7-428d-81a1-1657c4805596]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:31:01 user nova-compute[72081]: DEBUG os_brick.utils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] <== get_connector_properties: return (2151ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:31:02 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Updating existing volume attachment record: a21851bc-89bb-4047-846a-b003155bc1d5 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:31:02 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Successfully created port: ce9193f1-28a9-4c0f-95ae-e21f8140de19 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:31:02 user nova-compute[72081]: DEBUG nova.network.neutron [req-7ac4f5ea-23c0-4b50-b125-a98271eb666b req-0ebda4ba-8919-49cd-b01c-7c120a8b11a6 service nova] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:31:02 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:31:02 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:31:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:31:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.919s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:02 user nova-compute[72081]: DEBUG nova.network.neutron [req-7ac4f5ea-23c0-4b50-b125-a98271eb666b req-0ebda4ba-8919-49cd-b01c-7c120a8b11a6 service nova] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:31:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7ac4f5ea-23c0-4b50-b125-a98271eb666b req-0ebda4ba-8919-49cd-b01c-7c120a8b11a6 service nova] Releasing lock "refresh_cache-cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:31:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquired lock "refresh_cache-cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:31:02 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:31:02 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Successfully created port: ac401bfa-a662-4290-af6e-0a39e8fde61c {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Updating instance_info_cache with network_info: [{"id": "91c0bcc3-17a9-4444-bb3c-715e45188260", "address": "fa:16:3e:a8:9f:cb", "network": {"id": "07d0f97b-06f9-40b3-8794-e53924379d25", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-2076424457-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "86f7b42b161f41efb234f0ce70210a08", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap91c0bcc3-17", "ovs_interfaceid": "91c0bcc3-17a9-4444-bb3c-715e45188260", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Releasing lock "refresh_cache-cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Instance network_info: |[{"id": "91c0bcc3-17a9-4444-bb3c-715e45188260", "address": "fa:16:3e:a8:9f:cb", "network": {"id": "07d0f97b-06f9-40b3-8794-e53924379d25", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-2076424457-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "86f7b42b161f41efb234f0ce70210a08", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap91c0bcc3-17", "ovs_interfaceid": "91c0bcc3-17a9-4444-bb3c-715e45188260", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Start _get_guest_xml network_info=[{"id": "91c0bcc3-17a9-4444-bb3c-715e45188260", "address": "fa:16:3e:a8:9f:cb", "network": {"id": "07d0f97b-06f9-40b3-8794-e53924379d25", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-2076424457-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "86f7b42b161f41efb234f0ce70210a08", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap91c0bcc3-17", "ovs_interfaceid": "91c0bcc3-17a9-4444-bb3c-715e45188260", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:31:03 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:31:03 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:30:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-1090048845',display_name='tempest-VolumesSnapshotTestJSON-instance-1090048845',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-1090048845',id=8,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='86f7b42b161f41efb234f0ce70210a08',ramdisk_id='',reservation_id='r-yh0xe0j4',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesSnapshotTestJSON-1513599190',owner_user_name='tempest-VolumesSnapshotTestJSON-1513599190-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:30:58Z,user_data=None,user_id='84ea3a8776a944178ba5a04bce4d9b05',uuid=cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "91c0bcc3-17a9-4444-bb3c-715e45188260", "address": "fa:16:3e:a8:9f:cb", "network": {"id": "07d0f97b-06f9-40b3-8794-e53924379d25", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-2076424457-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "86f7b42b161f41efb234f0ce70210a08", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap91c0bcc3-17", "ovs_interfaceid": "91c0bcc3-17a9-4444-bb3c-715e45188260", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Converting VIF {"id": "91c0bcc3-17a9-4444-bb3c-715e45188260", "address": "fa:16:3e:a8:9f:cb", "network": {"id": "07d0f97b-06f9-40b3-8794-e53924379d25", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-2076424457-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "86f7b42b161f41efb234f0ce70210a08", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap91c0bcc3-17", "ovs_interfaceid": "91c0bcc3-17a9-4444-bb3c-715e45188260", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:a8:9f:cb,bridge_name='br-int',has_traffic_filtering=True,id=91c0bcc3-17a9-4444-bb3c-715e45188260,network=Network(07d0f97b-06f9-40b3-8794-e53924379d25),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap91c0bcc3-17') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.objects.instance [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lazy-loading 'pci_devices' on Instance uuid cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Rebuilding the list of instances to heal {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:31:03 user nova-compute[72081]:   <uuid>cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0</uuid>
Apr 12 03:31:03 user nova-compute[72081]:   <name>instance-00000008</name>
Apr 12 03:31:03 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:31:03 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:31:03 user nova-compute[72081]:   <metadata>
Apr 12 03:31:03 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:31:03 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:31:03 user nova-compute[72081]:       <nova:name>tempest-VolumesSnapshotTestJSON-instance-1090048845</nova:name>
Apr 12 03:31:03 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:31:03</nova:creationTime>
Apr 12 03:31:03 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:31:03 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:31:03 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:31:03 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:31:03 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:31:03 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:31:03 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:31:03 user nova-compute[72081]:       <nova:owner>
Apr 12 03:31:03 user nova-compute[72081]:         <nova:user uuid="84ea3a8776a944178ba5a04bce4d9b05">tempest-VolumesSnapshotTestJSON-1513599190-project-member</nova:user>
Apr 12 03:31:03 user nova-compute[72081]:         <nova:project uuid="86f7b42b161f41efb234f0ce70210a08">tempest-VolumesSnapshotTestJSON-1513599190</nova:project>
Apr 12 03:31:03 user nova-compute[72081]:       </nova:owner>
Apr 12 03:31:03 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:31:03 user nova-compute[72081]:       <nova:ports>
Apr 12 03:31:03 user nova-compute[72081]:         <nova:port uuid="91c0bcc3-17a9-4444-bb3c-715e45188260">
Apr 12 03:31:03 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Apr 12 03:31:03 user nova-compute[72081]:         </nova:port>
Apr 12 03:31:03 user nova-compute[72081]:       </nova:ports>
Apr 12 03:31:03 user nova-compute[72081]:     </nova:instance>
Apr 12 03:31:03 user nova-compute[72081]:   </metadata>
Apr 12 03:31:03 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:31:03 user nova-compute[72081]:     <system>
Apr 12 03:31:03 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:31:03 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:31:03 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:31:03 user nova-compute[72081]:       <entry name="serial">cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0</entry>
Apr 12 03:31:03 user nova-compute[72081]:       <entry name="uuid">cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0</entry>
Apr 12 03:31:03 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:31:03 user nova-compute[72081]:     </system>
Apr 12 03:31:03 user nova-compute[72081]:   </sysinfo>
Apr 12 03:31:03 user nova-compute[72081]:   <os>
Apr 12 03:31:03 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:31:03 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:31:03 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:31:03 user nova-compute[72081]:   </os>
Apr 12 03:31:03 user nova-compute[72081]:   <features>
Apr 12 03:31:03 user nova-compute[72081]:     <acpi/>
Apr 12 03:31:03 user nova-compute[72081]:     <apic/>
Apr 12 03:31:03 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:31:03 user nova-compute[72081]:   </features>
Apr 12 03:31:03 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:31:03 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:31:03 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:31:03 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:31:03 user nova-compute[72081]:   </clock>
Apr 12 03:31:03 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:31:03 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:31:03 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:31:03 user nova-compute[72081]:   </cpu>
Apr 12 03:31:03 user nova-compute[72081]:   <devices>
Apr 12 03:31:03 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:31:03 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:31:03 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0/disk"/>
Apr 12 03:31:03 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:31:03 user nova-compute[72081]:     </disk>
Apr 12 03:31:03 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:31:03 user nova-compute[72081]:       <mac address="fa:16:3e:a8:9f:cb"/>
Apr 12 03:31:03 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:31:03 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:31:03 user nova-compute[72081]:       <target dev="tap91c0bcc3-17"/>
Apr 12 03:31:03 user nova-compute[72081]:     </interface>
Apr 12 03:31:03 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:31:03 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0/console.log" append="off"/>
Apr 12 03:31:03 user nova-compute[72081]:     </serial>
Apr 12 03:31:03 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:31:03 user nova-compute[72081]:     <video>
Apr 12 03:31:03 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:31:03 user nova-compute[72081]:     </video>
Apr 12 03:31:03 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:31:03 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:31:03 user nova-compute[72081]:     </rng>
Apr 12 03:31:03 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:31:03 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:31:03 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:31:03 user nova-compute[72081]:     </memballoon>
Apr 12 03:31:03 user nova-compute[72081]:   </devices>
Apr 12 03:31:03 user nova-compute[72081]: </domain>
Apr 12 03:31:03 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:30:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-1090048845',display_name='tempest-VolumesSnapshotTestJSON-instance-1090048845',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-1090048845',id=8,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='86f7b42b161f41efb234f0ce70210a08',ramdisk_id='',reservation_id='r-yh0xe0j4',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesSnapshotTestJSON-1513599190',owner_user_name='tempest-VolumesSnapshotTestJSON-1513599190-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:30:58Z,user_data=None,user_id='84ea3a8776a944178ba5a04bce4d9b05',uuid=cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "91c0bcc3-17a9-4444-bb3c-715e45188260", "address": "fa:16:3e:a8:9f:cb", "network": {"id": "07d0f97b-06f9-40b3-8794-e53924379d25", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-2076424457-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "86f7b42b161f41efb234f0ce70210a08", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap91c0bcc3-17", "ovs_interfaceid": "91c0bcc3-17a9-4444-bb3c-715e45188260", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Converting VIF {"id": "91c0bcc3-17a9-4444-bb3c-715e45188260", "address": "fa:16:3e:a8:9f:cb", "network": {"id": "07d0f97b-06f9-40b3-8794-e53924379d25", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-2076424457-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "86f7b42b161f41efb234f0ce70210a08", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap91c0bcc3-17", "ovs_interfaceid": "91c0bcc3-17a9-4444-bb3c-715e45188260", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:a8:9f:cb,bridge_name='br-int',has_traffic_filtering=True,id=91c0bcc3-17a9-4444-bb3c-715e45188260,network=Network(07d0f97b-06f9-40b3-8794-e53924379d25),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap91c0bcc3-17') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG os_vif [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:a8:9f:cb,bridge_name='br-int',has_traffic_filtering=True,id=91c0bcc3-17a9-4444-bb3c-715e45188260,network=Network(07d0f97b-06f9-40b3-8794-e53924379d25),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap91c0bcc3-17') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap91c0bcc3-17, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap91c0bcc3-17, col_values=(('external_ids', {'iface-id': '91c0bcc3-17a9-4444-bb3c-715e45188260', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:a8:9f:cb', 'vm-uuid': 'cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Skipping network cache update for instance because it is Building. {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9805}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Skipping network cache update for instance because it is Building. {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9805}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:03 user nova-compute[72081]: INFO os_vif [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:a8:9f:cb,bridge_name='br-int',has_traffic_filtering=True,id=91c0bcc3-17a9-4444-bb3c-715e45188260,network=Network(07d0f97b-06f9-40b3-8794-e53924379d25),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap91c0bcc3-17')
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] No VIF found with MAC fa:16:3e:a8:9f:cb, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-56834002-7593-4592-a86d-23bb90d00be0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-56834002-7593-4592-a86d-23bb90d00be0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lazy-loading 'info_cache' on Instance uuid 56834002-7593-4592-a86d-23bb90d00be0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:03 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:31:03 user nova-compute[72081]: INFO nova.compute.claims [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Claim successful on node user
Apr 12 03:31:04 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.574s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:31:04 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:31:04 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Successfully created port: 4e57de22-e876-4bfe-bd07-a1003afd2295 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:31:04 user nova-compute[72081]: INFO nova.virt.block_device [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Booting with volume 6a6c8492-c29c-49f6-954b-a9381cd886e1 at /dev/vda
Apr 12 03:31:04 user nova-compute[72081]: DEBUG os_brick.utils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.012s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[136436ce-ecb5-4775-86e5-9dfe1a1559a8]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[fdf73bc7-1f61-42f6-8361-356aa74b5ced]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d50d0175-051d-47ff-8d76-f6bf89069105]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[653725f1-fde2-4b2b-bb37-d73e58c8a687]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Updating instance_info_cache with network_info: [{"id": "8f7b221a-25ae-41f9-847d-5aedf2d93a71", "address": "fa:16:3e:e7:22:1b", "network": {"id": "1a483654-b740-4544-84c9-53a41c11fa88", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1141588183-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "83a7ef3f97fd47849973d4e4482fb5f9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f7b221a-25", "ovs_interfaceid": "8f7b221a-25ae-41f9-847d-5aedf2d93a71", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-56834002-7593-4592-a86d-23bb90d00be0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:31:04 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:31:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9510442c-c87e-416d-ade3-ea7002c0369e]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG os_brick.utils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] <== get_connector_properties: return (151ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Updating existing volume attachment record: a9d33d1e-bb7e-42e3-a425-c102c25e85ba {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG nova.policy [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd29b6ed431d64bcab02c2348feab9aa6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd808e7e237f4c7490abaff5db0e3529', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:05 user nova-compute[72081]: DEBUG nova.compute.manager [req-5bd2db16-3941-4801-a525-d1b07c8ce021 req-72ba80a4-82a6-4ebd-a44c-f2d2efe439ff service nova] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Received event network-vif-plugged-91c0bcc3-17a9-4444-bb3c-715e45188260 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5bd2db16-3941-4801-a525-d1b07c8ce021 req-72ba80a4-82a6-4ebd-a44c-f2d2efe439ff service nova] Acquiring lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5bd2db16-3941-4801-a525-d1b07c8ce021 req-72ba80a4-82a6-4ebd-a44c-f2d2efe439ff service nova] Lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5bd2db16-3941-4801-a525-d1b07c8ce021 req-72ba80a4-82a6-4ebd-a44c-f2d2efe439ff service nova] Lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:05 user nova-compute[72081]: DEBUG nova.compute.manager [req-5bd2db16-3941-4801-a525-d1b07c8ce021 req-72ba80a4-82a6-4ebd-a44c-f2d2efe439ff service nova] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] No waiting events found dispatching network-vif-plugged-91c0bcc3-17a9-4444-bb3c-715e45188260 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:05 user nova-compute[72081]: WARNING nova.compute.manager [req-5bd2db16-3941-4801-a525-d1b07c8ce021 req-72ba80a4-82a6-4ebd-a44c-f2d2efe439ff service nova] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Received unexpected event network-vif-plugged-91c0bcc3-17a9-4444-bb3c-715e45188260 for instance with vm_state building and task_state spawning.
Apr 12 03:31:06 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Successfully created port: 8248913f-46c8-4f3d-ab47-0203cb267169 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:31:06 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Successfully created port: 33b2e7fb-520f-46d4-be86-e492aff77d4b {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:31:06 user nova-compute[72081]: INFO nova.virt.block_device [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Booting with volume 19d5feb6-1f29-4d61-8901-3b445f0c26fd at /dev/vdb
Apr 12 03:31:07 user nova-compute[72081]: DEBUG os_brick.utils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[253c99bd-4447-481a-b7ae-8e77e0c40745]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ac270473-a7c3-4811-a6df-4061107bded8]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4052b096-e387-4631-bc63-96c3863fe8f6]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd7507b2-17f2-45e7-a340-cf1200affcce]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:31:07 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:31:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[016925c8-8e4e-489a-810b-8f0dc0a39b02]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG os_brick.utils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] <== get_connector_properties: return (132ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Updating existing volume attachment record: 9fa884b4-08e2-4abf-bb3d-64835659b883 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270267.2134216, cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:31:07 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] VM Resumed (Lifecycle Event)
Apr 12 03:31:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:31:07 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Instance spawned successfully.
Apr 12 03:31:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:31:07 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:31:07 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270267.2145405, cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:31:07 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] VM Started (Lifecycle Event)
Apr 12 03:31:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:31:07 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:31:07 user nova-compute[72081]: INFO nova.compute.manager [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Took 9.19 seconds to spawn the instance on the hypervisor.
Apr 12 03:31:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-52489cd4-f574-44eb-8848-baa6d06d60da req-90b3200e-dfde-418a-9c88-3f4981524085 service nova] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Received event network-vif-plugged-91c0bcc3-17a9-4444-bb3c-715e45188260 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-52489cd4-f574-44eb-8848-baa6d06d60da req-90b3200e-dfde-418a-9c88-3f4981524085 service nova] Acquiring lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-52489cd4-f574-44eb-8848-baa6d06d60da req-90b3200e-dfde-418a-9c88-3f4981524085 service nova] Lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-52489cd4-f574-44eb-8848-baa6d06d60da req-90b3200e-dfde-418a-9c88-3f4981524085 service nova] Lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-52489cd4-f574-44eb-8848-baa6d06d60da req-90b3200e-dfde-418a-9c88-3f4981524085 service nova] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] No waiting events found dispatching network-vif-plugged-91c0bcc3-17a9-4444-bb3c-715e45188260 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:07 user nova-compute[72081]: WARNING nova.compute.manager [req-52489cd4-f574-44eb-8848-baa6d06d60da req-90b3200e-dfde-418a-9c88-3f4981524085 service nova] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Received unexpected event network-vif-plugged-91c0bcc3-17a9-4444-bb3c-715e45188260 for instance with vm_state building and task_state spawning.
Apr 12 03:31:07 user nova-compute[72081]: INFO nova.compute.manager [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Took 10.09 seconds to build instance.
Apr 12 03:31:07 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Successfully created port: 21860860-2a08-43dc-bda8-0a1d0fab0634 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:31:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a09205f-6280-478f-98d9-4e529975a15b tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 10.198s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:08 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Successfully created port: 91177380-5d77-4bed-9ab0-e9085fcb3d5a {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:31:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:08 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Successfully updated port: ce9193f1-28a9-4c0f-95ae-e21f8140de19 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:31:09 user nova-compute[72081]: INFO nova.virt.block_device [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Booting with volume 0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 at /dev/vdb
Apr 12 03:31:09 user nova-compute[72081]: DEBUG os_brick.utils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Successfully created port: 41626728-b21d-4ef8-9d86-5328d2feb3f9 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b1eddbcc-1ee4-4d31-b768-166a31a59f5d]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7d5ea9bd-3b4b-4c69-9c92-6efa28a6223b]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3687cffd-2bbe-41ac-ad9a-78e5ab296ee8]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1df360d5-c894-4109-9052-3fa1b8732af1]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:31:09 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:31:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[58a0d2f1-273f-450d-8425-c8dd85d46665]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG os_brick.utils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] <== get_connector_properties: return (121ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Updating existing volume attachment record: c671b848-20a3-459d-a7b2-d1ff1a072388 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-3962746f-4f69-4563-8fe6-cae5973f462f req-70e2480f-c56d-4c85-b47b-32c3154df6d4 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-changed-ce9193f1-28a9-4c0f-95ae-e21f8140de19 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-3962746f-4f69-4563-8fe6-cae5973f462f req-70e2480f-c56d-4c85-b47b-32c3154df6d4 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Refreshing instance network info cache due to event network-changed-ce9193f1-28a9-4c0f-95ae-e21f8140de19. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3962746f-4f69-4563-8fe6-cae5973f462f req-70e2480f-c56d-4c85-b47b-32c3154df6d4 service nova] Acquiring lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3962746f-4f69-4563-8fe6-cae5973f462f req-70e2480f-c56d-4c85-b47b-32c3154df6d4 service nova] Acquired lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG nova.network.neutron [req-3962746f-4f69-4563-8fe6-cae5973f462f req-70e2480f-c56d-4c85-b47b-32c3154df6d4 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Refreshing network info cache for port ce9193f1-28a9-4c0f-95ae-e21f8140de19 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:31:09 user nova-compute[72081]: DEBUG nova.network.neutron [req-3962746f-4f69-4563-8fe6-cae5973f462f req-70e2480f-c56d-4c85-b47b-32c3154df6d4 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:31:10 user nova-compute[72081]: DEBUG nova.network.neutron [req-3962746f-4f69-4563-8fe6-cae5973f462f req-70e2480f-c56d-4c85-b47b-32c3154df6d4 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:31:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3962746f-4f69-4563-8fe6-cae5973f462f req-70e2480f-c56d-4c85-b47b-32c3154df6d4 service nova] Releasing lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:31:10 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Successfully updated port: 835b5a2a-4d35-4d2a-9fdd-0c78b7def163 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:31:11 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Successfully created port: 6bb0c89d-0a60-4413-b09d-08d4d6ea3a52 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:31:11 user nova-compute[72081]: DEBUG nova.compute.manager [req-8b92eec3-2f0b-483e-a3e0-8c2711aa6a74 req-0b213fc9-8700-487f-aa0c-3baa19f5a563 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-changed-835b5a2a-4d35-4d2a-9fdd-0c78b7def163 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:11 user nova-compute[72081]: DEBUG nova.compute.manager [req-8b92eec3-2f0b-483e-a3e0-8c2711aa6a74 req-0b213fc9-8700-487f-aa0c-3baa19f5a563 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Refreshing instance network info cache due to event network-changed-835b5a2a-4d35-4d2a-9fdd-0c78b7def163. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:31:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8b92eec3-2f0b-483e-a3e0-8c2711aa6a74 req-0b213fc9-8700-487f-aa0c-3baa19f5a563 service nova] Acquiring lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:31:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8b92eec3-2f0b-483e-a3e0-8c2711aa6a74 req-0b213fc9-8700-487f-aa0c-3baa19f5a563 service nova] Acquired lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:31:11 user nova-compute[72081]: DEBUG nova.network.neutron [req-8b92eec3-2f0b-483e-a3e0-8c2711aa6a74 req-0b213fc9-8700-487f-aa0c-3baa19f5a563 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Refreshing network info cache for port 835b5a2a-4d35-4d2a-9fdd-0c78b7def163 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:31:11 user nova-compute[72081]: DEBUG nova.network.neutron [req-8b92eec3-2f0b-483e-a3e0-8c2711aa6a74 req-0b213fc9-8700-487f-aa0c-3baa19f5a563 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:31:12 user nova-compute[72081]: INFO nova.virt.block_device [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Booting with volume c73ac339-e533-4c8b-958d-ddf3e59dfb7d at /dev/vdc
Apr 12 03:31:12 user nova-compute[72081]: DEBUG os_brick.utils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c7ab6a17-19ca-40d1-bb6f-792abfaf3257]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f9e7060f-857e-4ddc-a461-d6ef7b9a74bd]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6652f54e-145b-4cb3-b34a-e9dddbea54fe]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[22e98fb9-8183-4758-806a-2fb9c35a78d7]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG nova.network.neutron [req-8b92eec3-2f0b-483e-a3e0-8c2711aa6a74 req-0b213fc9-8700-487f-aa0c-3baa19f5a563 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:31:12 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:31:12 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[560e7075-8009-48d8-806f-6263e92cfb8c]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG os_brick.utils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] <== get_connector_properties: return (116ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Updating existing volume attachment record: 63c70663-4aef-4386-9dc5-ce7986e28ef0 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8b92eec3-2f0b-483e-a3e0-8c2711aa6a74 req-0b213fc9-8700-487f-aa0c-3baa19f5a563 service nova] Releasing lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Successfully updated port: a1aaff41-e246-4887-89c6-a91163d4efd5 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Successfully created port: e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Acquiring lock "f1a0f3cf-1004-4ff9-9e41-d03291abbb9a" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Lock "f1a0f3cf-1004-4ff9-9e41-d03291abbb9a" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:12 user nova-compute[72081]: DEBUG nova.compute.manager [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:31:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:13 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:31:13 user nova-compute[72081]: INFO nova.compute.claims [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Claim successful on node user
Apr 12 03:31:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:13 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:31:13 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:31:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.457s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:13 user nova-compute[72081]: DEBUG nova.compute.manager [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:31:13 user nova-compute[72081]: DEBUG nova.compute.manager [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:31:13 user nova-compute[72081]: DEBUG nova.network.neutron [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:31:13 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:31:13 user nova-compute[72081]: DEBUG nova.compute.manager [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:31:13 user nova-compute[72081]: DEBUG nova.compute.manager [req-3dfbe5a3-d4e9-4855-9ec6-3d9011d3965a req-30be2019-9397-4b26-83aa-38305c48d989 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-changed-a1aaff41-e246-4887-89c6-a91163d4efd5 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:13 user nova-compute[72081]: DEBUG nova.compute.manager [req-3dfbe5a3-d4e9-4855-9ec6-3d9011d3965a req-30be2019-9397-4b26-83aa-38305c48d989 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Refreshing instance network info cache due to event network-changed-a1aaff41-e246-4887-89c6-a91163d4efd5. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:31:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3dfbe5a3-d4e9-4855-9ec6-3d9011d3965a req-30be2019-9397-4b26-83aa-38305c48d989 service nova] Acquiring lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:31:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3dfbe5a3-d4e9-4855-9ec6-3d9011d3965a req-30be2019-9397-4b26-83aa-38305c48d989 service nova] Acquired lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:31:13 user nova-compute[72081]: DEBUG nova.network.neutron [req-3dfbe5a3-d4e9-4855-9ec6-3d9011d3965a req-30be2019-9397-4b26-83aa-38305c48d989 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Refreshing network info cache for port a1aaff41-e246-4887-89c6-a91163d4efd5 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:31:13 user nova-compute[72081]: DEBUG nova.compute.manager [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:31:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:31:13 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Creating image(s)
Apr 12 03:31:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Acquiring lock "/opt/stack/data/nova/instances/f1a0f3cf-1004-4ff9-9e41-d03291abbb9a/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Lock "/opt/stack/data/nova/instances/f1a0f3cf-1004-4ff9-9e41-d03291abbb9a/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Lock "/opt/stack/data/nova/instances/f1a0f3cf-1004-4ff9-9e41-d03291abbb9a/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.015s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:13 user nova-compute[72081]: DEBUG nova.network.neutron [req-3dfbe5a3-d4e9-4855-9ec6-3d9011d3965a req-30be2019-9397-4b26-83aa-38305c48d989 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:31:13 user nova-compute[72081]: DEBUG nova.policy [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '1183761babfc4ba3b6f6febe28254c15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f71b2bab30884740b8e6d2c268c7635b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:31:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.144s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:13 user nova-compute[72081]: INFO nova.virt.block_device [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Booting with volume 6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 at /dev/vdc
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.141s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/f1a0f3cf-1004-4ff9-9e41-d03291abbb9a/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG os_brick.utils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/f1a0f3cf-1004-4ff9-9e41-d03291abbb9a/disk 1073741824" returned: 0 in 0.058s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.203s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2e2f6ce2-9b67-4dc3-9c5e-c0742ce00462]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[54c34253-f2ad-4088-a27f-8bb24ac2d497]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1b484c96-af25-4f12-8379-02329ca798da]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[efa43c58-4553-4713-a41a-ce0f93723f1e]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:31:14 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e37bf1f0-fb92-4452-82e4-2dd8fb2422c7]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG os_brick.utils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] <== get_connector_properties: return (172ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Updating existing volume attachment record: 4a6652f8-39d1-49dc-992d-b402a6900247 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.165s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Checking if we can resize image /opt/stack/data/nova/instances/f1a0f3cf-1004-4ff9-9e41-d03291abbb9a/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f1a0f3cf-1004-4ff9-9e41-d03291abbb9a/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG nova.network.neutron [req-3dfbe5a3-d4e9-4855-9ec6-3d9011d3965a req-30be2019-9397-4b26-83aa-38305c48d989 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f1a0f3cf-1004-4ff9-9e41-d03291abbb9a/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Cannot resize image /opt/stack/data/nova/instances/f1a0f3cf-1004-4ff9-9e41-d03291abbb9a/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG nova.objects.instance [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Lazy-loading 'migration_context' on Instance uuid f1a0f3cf-1004-4ff9-9e41-d03291abbb9a {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3dfbe5a3-d4e9-4855-9ec6-3d9011d3965a req-30be2019-9397-4b26-83aa-38305c48d989 service nova] Releasing lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Ensure instance console log exists: /opt/stack/data/nova/instances/f1a0f3cf-1004-4ff9-9e41-d03291abbb9a/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Successfully updated port: ac401bfa-a662-4290-af6e-0a39e8fde61c {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:31:14 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Successfully updated port: 33b2e7fb-520f-46d4-be86-e492aff77d4b {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:31:15 user nova-compute[72081]: DEBUG nova.compute.manager [req-285e088b-a101-41dd-9825-a5c577916b8e req-7448c8ce-aa12-4816-ac7d-57537bfc4c03 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-changed-ac401bfa-a662-4290-af6e-0a39e8fde61c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:15 user nova-compute[72081]: DEBUG nova.compute.manager [req-285e088b-a101-41dd-9825-a5c577916b8e req-7448c8ce-aa12-4816-ac7d-57537bfc4c03 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Refreshing instance network info cache due to event network-changed-ac401bfa-a662-4290-af6e-0a39e8fde61c. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:31:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-285e088b-a101-41dd-9825-a5c577916b8e req-7448c8ce-aa12-4816-ac7d-57537bfc4c03 service nova] Acquiring lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:31:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-285e088b-a101-41dd-9825-a5c577916b8e req-7448c8ce-aa12-4816-ac7d-57537bfc4c03 service nova] Acquired lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:31:15 user nova-compute[72081]: DEBUG nova.network.neutron [req-285e088b-a101-41dd-9825-a5c577916b8e req-7448c8ce-aa12-4816-ac7d-57537bfc4c03 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Refreshing network info cache for port ac401bfa-a662-4290-af6e-0a39e8fde61c {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:31:15 user nova-compute[72081]: DEBUG nova.network.neutron [req-285e088b-a101-41dd-9825-a5c577916b8e req-7448c8ce-aa12-4816-ac7d-57537bfc4c03 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:31:15 user nova-compute[72081]: DEBUG nova.network.neutron [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Successfully created port: e9d42f8f-a0eb-4058-83e3-f5014b577443 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:31:16 user nova-compute[72081]: DEBUG nova.network.neutron [req-285e088b-a101-41dd-9825-a5c577916b8e req-7448c8ce-aa12-4816-ac7d-57537bfc4c03 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:31:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-285e088b-a101-41dd-9825-a5c577916b8e req-7448c8ce-aa12-4816-ac7d-57537bfc4c03 service nova] Releasing lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:31:16 user nova-compute[72081]: DEBUG nova.compute.manager [req-285e088b-a101-41dd-9825-a5c577916b8e req-7448c8ce-aa12-4816-ac7d-57537bfc4c03 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-changed-33b2e7fb-520f-46d4-be86-e492aff77d4b {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:16 user nova-compute[72081]: DEBUG nova.compute.manager [req-285e088b-a101-41dd-9825-a5c577916b8e req-7448c8ce-aa12-4816-ac7d-57537bfc4c03 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Refreshing instance network info cache due to event network-changed-33b2e7fb-520f-46d4-be86-e492aff77d4b. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:31:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-285e088b-a101-41dd-9825-a5c577916b8e req-7448c8ce-aa12-4816-ac7d-57537bfc4c03 service nova] Acquiring lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:31:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-285e088b-a101-41dd-9825-a5c577916b8e req-7448c8ce-aa12-4816-ac7d-57537bfc4c03 service nova] Acquired lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:31:16 user nova-compute[72081]: DEBUG nova.network.neutron [req-285e088b-a101-41dd-9825-a5c577916b8e req-7448c8ce-aa12-4816-ac7d-57537bfc4c03 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Refreshing network info cache for port 33b2e7fb-520f-46d4-be86-e492aff77d4b {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:31:16 user nova-compute[72081]: DEBUG nova.network.neutron [req-285e088b-a101-41dd-9825-a5c577916b8e req-7448c8ce-aa12-4816-ac7d-57537bfc4c03 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:31:16 user nova-compute[72081]: DEBUG nova.network.neutron [req-285e088b-a101-41dd-9825-a5c577916b8e req-7448c8ce-aa12-4816-ac7d-57537bfc4c03 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:31:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-285e088b-a101-41dd-9825-a5c577916b8e req-7448c8ce-aa12-4816-ac7d-57537bfc4c03 service nova] Releasing lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:31:16 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Successfully updated port: 4e57de22-e876-4bfe-bd07-a1003afd2295 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:31:16 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Successfully updated port: 7740f806-8319-484b-8b0f-bffd8c5fe532 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:31:17 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:31:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:31:17 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Creating image(s)
Apr 12 03:31:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Did not create local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 12 03:31:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Ensure instance console log exists: /opt/stack/data/nova/instances/7119b8b3-291b-4868-820e-d7bd3dd3ada4/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:31:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:17 user nova-compute[72081]: DEBUG nova.network.neutron [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Successfully updated port: e9d42f8f-a0eb-4058-83e3-f5014b577443 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:31:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Acquiring lock "refresh_cache-f1a0f3cf-1004-4ff9-9e41-d03291abbb9a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:31:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Acquired lock "refresh_cache-f1a0f3cf-1004-4ff9-9e41-d03291abbb9a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:31:17 user nova-compute[72081]: DEBUG nova.network.neutron [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:31:17 user nova-compute[72081]: DEBUG nova.network.neutron [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:31:17 user nova-compute[72081]: DEBUG nova.compute.manager [req-3d8454ba-a316-4b16-9625-264bf7b01586 req-c83bbb49-e4dd-435f-88eb-8cb282886f53 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-changed-4e57de22-e876-4bfe-bd07-a1003afd2295 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:17 user nova-compute[72081]: DEBUG nova.compute.manager [req-3d8454ba-a316-4b16-9625-264bf7b01586 req-c83bbb49-e4dd-435f-88eb-8cb282886f53 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Refreshing instance network info cache due to event network-changed-4e57de22-e876-4bfe-bd07-a1003afd2295. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:31:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3d8454ba-a316-4b16-9625-264bf7b01586 req-c83bbb49-e4dd-435f-88eb-8cb282886f53 service nova] Acquiring lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:31:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3d8454ba-a316-4b16-9625-264bf7b01586 req-c83bbb49-e4dd-435f-88eb-8cb282886f53 service nova] Acquired lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:31:17 user nova-compute[72081]: DEBUG nova.network.neutron [req-3d8454ba-a316-4b16-9625-264bf7b01586 req-c83bbb49-e4dd-435f-88eb-8cb282886f53 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Refreshing network info cache for port 4e57de22-e876-4bfe-bd07-a1003afd2295 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:31:17 user nova-compute[72081]: DEBUG nova.network.neutron [req-3d8454ba-a316-4b16-9625-264bf7b01586 req-c83bbb49-e4dd-435f-88eb-8cb282886f53 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.network.neutron [req-3d8454ba-a316-4b16-9625-264bf7b01586 req-c83bbb49-e4dd-435f-88eb-8cb282886f53 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3d8454ba-a316-4b16-9625-264bf7b01586 req-c83bbb49-e4dd-435f-88eb-8cb282886f53 service nova] Releasing lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.compute.manager [req-3d8454ba-a316-4b16-9625-264bf7b01586 req-c83bbb49-e4dd-435f-88eb-8cb282886f53 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-changed-7740f806-8319-484b-8b0f-bffd8c5fe532 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.compute.manager [req-3d8454ba-a316-4b16-9625-264bf7b01586 req-c83bbb49-e4dd-435f-88eb-8cb282886f53 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Refreshing instance network info cache due to event network-changed-7740f806-8319-484b-8b0f-bffd8c5fe532. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3d8454ba-a316-4b16-9625-264bf7b01586 req-c83bbb49-e4dd-435f-88eb-8cb282886f53 service nova] Acquiring lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3d8454ba-a316-4b16-9625-264bf7b01586 req-c83bbb49-e4dd-435f-88eb-8cb282886f53 service nova] Acquired lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.network.neutron [req-3d8454ba-a316-4b16-9625-264bf7b01586 req-c83bbb49-e4dd-435f-88eb-8cb282886f53 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Refreshing network info cache for port 7740f806-8319-484b-8b0f-bffd8c5fe532 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.network.neutron [req-3d8454ba-a316-4b16-9625-264bf7b01586 req-c83bbb49-e4dd-435f-88eb-8cb282886f53 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.network.neutron [req-3d8454ba-a316-4b16-9625-264bf7b01586 req-c83bbb49-e4dd-435f-88eb-8cb282886f53 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3d8454ba-a316-4b16-9625-264bf7b01586 req-c83bbb49-e4dd-435f-88eb-8cb282886f53 service nova] Releasing lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.compute.manager [req-3d8454ba-a316-4b16-9625-264bf7b01586 req-c83bbb49-e4dd-435f-88eb-8cb282886f53 service nova] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Received event network-changed-e9d42f8f-a0eb-4058-83e3-f5014b577443 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.compute.manager [req-3d8454ba-a316-4b16-9625-264bf7b01586 req-c83bbb49-e4dd-435f-88eb-8cb282886f53 service nova] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Refreshing instance network info cache due to event network-changed-e9d42f8f-a0eb-4058-83e3-f5014b577443. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3d8454ba-a316-4b16-9625-264bf7b01586 req-c83bbb49-e4dd-435f-88eb-8cb282886f53 service nova] Acquiring lock "refresh_cache-f1a0f3cf-1004-4ff9-9e41-d03291abbb9a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:31:18 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Creating image(s)
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Did not create local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Ensure instance console log exists: /opt/stack/data/nova/instances/43289ac5-3300-4e1f-8d07-2d66ee78e7ba/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Successfully updated port: 8248913f-46c8-4f3d-ab47-0203cb267169 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.network.neutron [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Updating instance_info_cache with network_info: [{"id": "e9d42f8f-a0eb-4058-83e3-f5014b577443", "address": "fa:16:3e:38:4c:25", "network": {"id": "d269b6dc-1cad-4872-9d0f-b4954e3c0aa0", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.139", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "e0a848f7aa0143d98afdd0e4e05fe87c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape9d42f8f-a0", "ovs_interfaceid": "e9d42f8f-a0eb-4058-83e3-f5014b577443", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Successfully updated port: be3de447-6d22-4fec-b962-0ebe772ed86f {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Releasing lock "refresh_cache-f1a0f3cf-1004-4ff9-9e41-d03291abbb9a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Instance network_info: |[{"id": "e9d42f8f-a0eb-4058-83e3-f5014b577443", "address": "fa:16:3e:38:4c:25", "network": {"id": "d269b6dc-1cad-4872-9d0f-b4954e3c0aa0", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.139", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "e0a848f7aa0143d98afdd0e4e05fe87c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape9d42f8f-a0", "ovs_interfaceid": "e9d42f8f-a0eb-4058-83e3-f5014b577443", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3d8454ba-a316-4b16-9625-264bf7b01586 req-c83bbb49-e4dd-435f-88eb-8cb282886f53 service nova] Acquired lock "refresh_cache-f1a0f3cf-1004-4ff9-9e41-d03291abbb9a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.network.neutron [req-3d8454ba-a316-4b16-9625-264bf7b01586 req-c83bbb49-e4dd-435f-88eb-8cb282886f53 service nova] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Refreshing network info cache for port e9d42f8f-a0eb-4058-83e3-f5014b577443 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Start _get_guest_xml network_info=[{"id": "e9d42f8f-a0eb-4058-83e3-f5014b577443", "address": "fa:16:3e:38:4c:25", "network": {"id": "d269b6dc-1cad-4872-9d0f-b4954e3c0aa0", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.139", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "e0a848f7aa0143d98afdd0e4e05fe87c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape9d42f8f-a0", "ovs_interfaceid": "e9d42f8f-a0eb-4058-83e3-f5014b577443", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:31:18 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:31:18 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:31:12Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesNegativeTest-instance-1006783758',display_name='tempest-VolumesNegativeTest-instance-1006783758',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesnegativetest-instance-1006783758',id=11,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f71b2bab30884740b8e6d2c268c7635b',ramdisk_id='',reservation_id='r-ss0xrpn1',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesNegativeTest-833717359',owner_user_name='tempest-VolumesNegativeTest-833717359-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:14Z,user_data=None,user_id='1183761babfc4ba3b6f6febe28254c15',uuid=f1a0f3cf-1004-4ff9-9e41-d03291abbb9a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e9d42f8f-a0eb-4058-83e3-f5014b577443", "address": "fa:16:3e:38:4c:25", "network": {"id": "d269b6dc-1cad-4872-9d0f-b4954e3c0aa0", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.139", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "e0a848f7aa0143d98afdd0e4e05fe87c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape9d42f8f-a0", "ovs_interfaceid": "e9d42f8f-a0eb-4058-83e3-f5014b577443", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Converting VIF {"id": "e9d42f8f-a0eb-4058-83e3-f5014b577443", "address": "fa:16:3e:38:4c:25", "network": {"id": "d269b6dc-1cad-4872-9d0f-b4954e3c0aa0", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.139", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "e0a848f7aa0143d98afdd0e4e05fe87c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape9d42f8f-a0", "ovs_interfaceid": "e9d42f8f-a0eb-4058-83e3-f5014b577443", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:38:4c:25,bridge_name='br-int',has_traffic_filtering=True,id=e9d42f8f-a0eb-4058-83e3-f5014b577443,network=Network(d269b6dc-1cad-4872-9d0f-b4954e3c0aa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape9d42f8f-a0') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.objects.instance [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Lazy-loading 'pci_devices' on Instance uuid f1a0f3cf-1004-4ff9-9e41-d03291abbb9a {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:31:18 user nova-compute[72081]:   <uuid>f1a0f3cf-1004-4ff9-9e41-d03291abbb9a</uuid>
Apr 12 03:31:18 user nova-compute[72081]:   <name>instance-0000000b</name>
Apr 12 03:31:18 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:31:18 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:31:18 user nova-compute[72081]:   <metadata>
Apr 12 03:31:18 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:31:18 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:31:18 user nova-compute[72081]:       <nova:name>tempest-VolumesNegativeTest-instance-1006783758</nova:name>
Apr 12 03:31:18 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:31:18</nova:creationTime>
Apr 12 03:31:18 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:31:18 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:31:18 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:31:18 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:31:18 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:31:18 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:31:18 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:31:18 user nova-compute[72081]:       <nova:owner>
Apr 12 03:31:18 user nova-compute[72081]:         <nova:user uuid="1183761babfc4ba3b6f6febe28254c15">tempest-VolumesNegativeTest-833717359-project-member</nova:user>
Apr 12 03:31:18 user nova-compute[72081]:         <nova:project uuid="f71b2bab30884740b8e6d2c268c7635b">tempest-VolumesNegativeTest-833717359</nova:project>
Apr 12 03:31:18 user nova-compute[72081]:       </nova:owner>
Apr 12 03:31:18 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:31:18 user nova-compute[72081]:       <nova:ports>
Apr 12 03:31:18 user nova-compute[72081]:         <nova:port uuid="e9d42f8f-a0eb-4058-83e3-f5014b577443">
Apr 12 03:31:18 user nova-compute[72081]:           <nova:ip type="fixed" address="192.168.233.139" ipVersion="4"/>
Apr 12 03:31:18 user nova-compute[72081]:         </nova:port>
Apr 12 03:31:18 user nova-compute[72081]:       </nova:ports>
Apr 12 03:31:18 user nova-compute[72081]:     </nova:instance>
Apr 12 03:31:18 user nova-compute[72081]:   </metadata>
Apr 12 03:31:18 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:31:18 user nova-compute[72081]:     <system>
Apr 12 03:31:18 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:31:18 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:31:18 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:31:18 user nova-compute[72081]:       <entry name="serial">f1a0f3cf-1004-4ff9-9e41-d03291abbb9a</entry>
Apr 12 03:31:18 user nova-compute[72081]:       <entry name="uuid">f1a0f3cf-1004-4ff9-9e41-d03291abbb9a</entry>
Apr 12 03:31:18 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:31:18 user nova-compute[72081]:     </system>
Apr 12 03:31:18 user nova-compute[72081]:   </sysinfo>
Apr 12 03:31:18 user nova-compute[72081]:   <os>
Apr 12 03:31:18 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:31:18 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:31:18 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:31:18 user nova-compute[72081]:   </os>
Apr 12 03:31:18 user nova-compute[72081]:   <features>
Apr 12 03:31:18 user nova-compute[72081]:     <acpi/>
Apr 12 03:31:18 user nova-compute[72081]:     <apic/>
Apr 12 03:31:18 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:31:18 user nova-compute[72081]:   </features>
Apr 12 03:31:18 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:31:18 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:31:18 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:31:18 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:31:18 user nova-compute[72081]:   </clock>
Apr 12 03:31:18 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:31:18 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:31:18 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:31:18 user nova-compute[72081]:   </cpu>
Apr 12 03:31:18 user nova-compute[72081]:   <devices>
Apr 12 03:31:18 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:31:18 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:31:18 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/f1a0f3cf-1004-4ff9-9e41-d03291abbb9a/disk"/>
Apr 12 03:31:18 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:31:18 user nova-compute[72081]:     </disk>
Apr 12 03:31:18 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:31:18 user nova-compute[72081]:       <mac address="fa:16:3e:38:4c:25"/>
Apr 12 03:31:18 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:31:18 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:31:18 user nova-compute[72081]:       <target dev="tape9d42f8f-a0"/>
Apr 12 03:31:18 user nova-compute[72081]:     </interface>
Apr 12 03:31:18 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:31:18 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/f1a0f3cf-1004-4ff9-9e41-d03291abbb9a/console.log" append="off"/>
Apr 12 03:31:18 user nova-compute[72081]:     </serial>
Apr 12 03:31:18 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:31:18 user nova-compute[72081]:     <video>
Apr 12 03:31:18 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:31:18 user nova-compute[72081]:     </video>
Apr 12 03:31:18 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:31:18 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:31:18 user nova-compute[72081]:     </rng>
Apr 12 03:31:18 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:31:18 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:31:18 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:31:18 user nova-compute[72081]:     </memballoon>
Apr 12 03:31:18 user nova-compute[72081]:   </devices>
Apr 12 03:31:18 user nova-compute[72081]: </domain>
Apr 12 03:31:18 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:31:12Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesNegativeTest-instance-1006783758',display_name='tempest-VolumesNegativeTest-instance-1006783758',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesnegativetest-instance-1006783758',id=11,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f71b2bab30884740b8e6d2c268c7635b',ramdisk_id='',reservation_id='r-ss0xrpn1',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesNegativeTest-833717359',owner_user_name='tempest-VolumesNegativeTest-833717359-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:14Z,user_data=None,user_id='1183761babfc4ba3b6f6febe28254c15',uuid=f1a0f3cf-1004-4ff9-9e41-d03291abbb9a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e9d42f8f-a0eb-4058-83e3-f5014b577443", "address": "fa:16:3e:38:4c:25", "network": {"id": "d269b6dc-1cad-4872-9d0f-b4954e3c0aa0", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.139", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "e0a848f7aa0143d98afdd0e4e05fe87c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape9d42f8f-a0", "ovs_interfaceid": "e9d42f8f-a0eb-4058-83e3-f5014b577443", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Converting VIF {"id": "e9d42f8f-a0eb-4058-83e3-f5014b577443", "address": "fa:16:3e:38:4c:25", "network": {"id": "d269b6dc-1cad-4872-9d0f-b4954e3c0aa0", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.139", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "e0a848f7aa0143d98afdd0e4e05fe87c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape9d42f8f-a0", "ovs_interfaceid": "e9d42f8f-a0eb-4058-83e3-f5014b577443", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:38:4c:25,bridge_name='br-int',has_traffic_filtering=True,id=e9d42f8f-a0eb-4058-83e3-f5014b577443,network=Network(d269b6dc-1cad-4872-9d0f-b4954e3c0aa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape9d42f8f-a0') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG os_vif [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:38:4c:25,bridge_name='br-int',has_traffic_filtering=True,id=e9d42f8f-a0eb-4058-83e3-f5014b577443,network=Network(d269b6dc-1cad-4872-9d0f-b4954e3c0aa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape9d42f8f-a0') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tape9d42f8f-a0, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tape9d42f8f-a0, col_values=(('external_ids', {'iface-id': 'e9d42f8f-a0eb-4058-83e3-f5014b577443', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:38:4c:25', 'vm-uuid': 'f1a0f3cf-1004-4ff9-9e41-d03291abbb9a'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:18 user nova-compute[72081]: INFO os_vif [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:38:4c:25,bridge_name='br-int',has_traffic_filtering=True,id=e9d42f8f-a0eb-4058-83e3-f5014b577443,network=Network(d269b6dc-1cad-4872-9d0f-b4954e3c0aa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape9d42f8f-a0')
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:31:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] No VIF found with MAC fa:16:3e:38:4c:25, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:31:19 user nova-compute[72081]: DEBUG nova.network.neutron [req-3d8454ba-a316-4b16-9625-264bf7b01586 req-c83bbb49-e4dd-435f-88eb-8cb282886f53 service nova] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Updated VIF entry in instance network info cache for port e9d42f8f-a0eb-4058-83e3-f5014b577443. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:31:19 user nova-compute[72081]: DEBUG nova.network.neutron [req-3d8454ba-a316-4b16-9625-264bf7b01586 req-c83bbb49-e4dd-435f-88eb-8cb282886f53 service nova] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Updating instance_info_cache with network_info: [{"id": "e9d42f8f-a0eb-4058-83e3-f5014b577443", "address": "fa:16:3e:38:4c:25", "network": {"id": "d269b6dc-1cad-4872-9d0f-b4954e3c0aa0", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.139", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "e0a848f7aa0143d98afdd0e4e05fe87c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape9d42f8f-a0", "ovs_interfaceid": "e9d42f8f-a0eb-4058-83e3-f5014b577443", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:31:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3d8454ba-a316-4b16-9625-264bf7b01586 req-c83bbb49-e4dd-435f-88eb-8cb282886f53 service nova] Releasing lock "refresh_cache-f1a0f3cf-1004-4ff9-9e41-d03291abbb9a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:31:19 user nova-compute[72081]: DEBUG nova.compute.manager [req-f680e68c-3809-4af3-aef3-a6acb9b332c8 req-276e4ae3-718a-4a69-9c50-54e489f74ab4 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-changed-8248913f-46c8-4f3d-ab47-0203cb267169 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:19 user nova-compute[72081]: DEBUG nova.compute.manager [req-f680e68c-3809-4af3-aef3-a6acb9b332c8 req-276e4ae3-718a-4a69-9c50-54e489f74ab4 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Refreshing instance network info cache due to event network-changed-8248913f-46c8-4f3d-ab47-0203cb267169. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:31:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f680e68c-3809-4af3-aef3-a6acb9b332c8 req-276e4ae3-718a-4a69-9c50-54e489f74ab4 service nova] Acquiring lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:31:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f680e68c-3809-4af3-aef3-a6acb9b332c8 req-276e4ae3-718a-4a69-9c50-54e489f74ab4 service nova] Acquired lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:31:19 user nova-compute[72081]: DEBUG nova.network.neutron [req-f680e68c-3809-4af3-aef3-a6acb9b332c8 req-276e4ae3-718a-4a69-9c50-54e489f74ab4 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Refreshing network info cache for port 8248913f-46c8-4f3d-ab47-0203cb267169 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:31:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:20 user nova-compute[72081]: DEBUG nova.network.neutron [req-f680e68c-3809-4af3-aef3-a6acb9b332c8 req-276e4ae3-718a-4a69-9c50-54e489f74ab4 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:31:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:20 user nova-compute[72081]: DEBUG nova.compute.manager [req-b5437944-8a98-4308-a0f0-109a59fff3eb req-ee72359c-ba30-47e5-8cc4-fc6d35cc888a service nova] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Received event network-vif-plugged-e9d42f8f-a0eb-4058-83e3-f5014b577443 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b5437944-8a98-4308-a0f0-109a59fff3eb req-ee72359c-ba30-47e5-8cc4-fc6d35cc888a service nova] Acquiring lock "f1a0f3cf-1004-4ff9-9e41-d03291abbb9a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b5437944-8a98-4308-a0f0-109a59fff3eb req-ee72359c-ba30-47e5-8cc4-fc6d35cc888a service nova] Lock "f1a0f3cf-1004-4ff9-9e41-d03291abbb9a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b5437944-8a98-4308-a0f0-109a59fff3eb req-ee72359c-ba30-47e5-8cc4-fc6d35cc888a service nova] Lock "f1a0f3cf-1004-4ff9-9e41-d03291abbb9a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:20 user nova-compute[72081]: DEBUG nova.compute.manager [req-b5437944-8a98-4308-a0f0-109a59fff3eb req-ee72359c-ba30-47e5-8cc4-fc6d35cc888a service nova] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] No waiting events found dispatching network-vif-plugged-e9d42f8f-a0eb-4058-83e3-f5014b577443 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:20 user nova-compute[72081]: WARNING nova.compute.manager [req-b5437944-8a98-4308-a0f0-109a59fff3eb req-ee72359c-ba30-47e5-8cc4-fc6d35cc888a service nova] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Received unexpected event network-vif-plugged-e9d42f8f-a0eb-4058-83e3-f5014b577443 for instance with vm_state building and task_state spawning.
Apr 12 03:31:20 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Successfully updated port: 21860860-2a08-43dc-bda8-0a1d0fab0634 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:31:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Acquiring lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:31:20 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Successfully updated port: 91177380-5d77-4bed-9ab0-e9085fcb3d5a {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:31:20 user nova-compute[72081]: DEBUG nova.network.neutron [req-f680e68c-3809-4af3-aef3-a6acb9b332c8 req-276e4ae3-718a-4a69-9c50-54e489f74ab4 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:31:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f680e68c-3809-4af3-aef3-a6acb9b332c8 req-276e4ae3-718a-4a69-9c50-54e489f74ab4 service nova] Releasing lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:31:20 user nova-compute[72081]: DEBUG nova.compute.manager [req-f680e68c-3809-4af3-aef3-a6acb9b332c8 req-276e4ae3-718a-4a69-9c50-54e489f74ab4 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-changed-be3de447-6d22-4fec-b962-0ebe772ed86f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:20 user nova-compute[72081]: DEBUG nova.compute.manager [req-f680e68c-3809-4af3-aef3-a6acb9b332c8 req-276e4ae3-718a-4a69-9c50-54e489f74ab4 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Refreshing instance network info cache due to event network-changed-be3de447-6d22-4fec-b962-0ebe772ed86f. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:31:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f680e68c-3809-4af3-aef3-a6acb9b332c8 req-276e4ae3-718a-4a69-9c50-54e489f74ab4 service nova] Acquiring lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:31:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f680e68c-3809-4af3-aef3-a6acb9b332c8 req-276e4ae3-718a-4a69-9c50-54e489f74ab4 service nova] Acquired lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:31:20 user nova-compute[72081]: DEBUG nova.network.neutron [req-f680e68c-3809-4af3-aef3-a6acb9b332c8 req-276e4ae3-718a-4a69-9c50-54e489f74ab4 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Refreshing network info cache for port be3de447-6d22-4fec-b962-0ebe772ed86f {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:31:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Acquired lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:31:20 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:31:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:20 user nova-compute[72081]: DEBUG nova.network.neutron [req-f680e68c-3809-4af3-aef3-a6acb9b332c8 req-276e4ae3-718a-4a69-9c50-54e489f74ab4 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:31:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:20 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:31:21 user nova-compute[72081]: DEBUG nova.network.neutron [req-f680e68c-3809-4af3-aef3-a6acb9b332c8 req-276e4ae3-718a-4a69-9c50-54e489f74ab4 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:31:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f680e68c-3809-4af3-aef3-a6acb9b332c8 req-276e4ae3-718a-4a69-9c50-54e489f74ab4 service nova] Releasing lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:31:21 user nova-compute[72081]: DEBUG nova.compute.manager [req-de5a623d-7a92-4567-be13-724a403a9a5c req-167d98d8-9e72-460b-ba3b-4afb1f3742ab service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-changed-21860860-2a08-43dc-bda8-0a1d0fab0634 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:21 user nova-compute[72081]: DEBUG nova.compute.manager [req-de5a623d-7a92-4567-be13-724a403a9a5c req-167d98d8-9e72-460b-ba3b-4afb1f3742ab service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Refreshing instance network info cache due to event network-changed-21860860-2a08-43dc-bda8-0a1d0fab0634. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:31:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-de5a623d-7a92-4567-be13-724a403a9a5c req-167d98d8-9e72-460b-ba3b-4afb1f3742ab service nova] Acquiring lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:31:21 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Successfully updated port: 41626728-b21d-4ef8-9d86-5328d2feb3f9 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:31:22 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270282.2977989, f1a0f3cf-1004-4ff9-9e41-d03291abbb9a => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:31:22 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] VM Resumed (Lifecycle Event)
Apr 12 03:31:22 user nova-compute[72081]: DEBUG nova.compute.manager [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:31:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:31:22 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Instance spawned successfully.
Apr 12 03:31:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:31:22 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:31:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:31:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:31:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:31:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:31:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:31:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:31:22 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:31:22 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:31:22 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270282.3007164, f1a0f3cf-1004-4ff9-9e41-d03291abbb9a => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:31:22 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] VM Started (Lifecycle Event)
Apr 12 03:31:22 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:31:22 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:31:22 user nova-compute[72081]: INFO nova.compute.manager [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Took 8.71 seconds to spawn the instance on the hypervisor.
Apr 12 03:31:22 user nova-compute[72081]: DEBUG nova.compute.manager [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:31:22 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:31:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:22 user nova-compute[72081]: INFO nova.compute.manager [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Took 9.50 seconds to build instance.
Apr 12 03:31:22 user nova-compute[72081]: DEBUG nova.compute.manager [req-5fbfc7e4-416b-4694-8e99-2ed00a8df3b6 req-fec353e5-4ac5-4e10-a69e-3b9c7d8a38bf service nova] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Received event network-vif-plugged-e9d42f8f-a0eb-4058-83e3-f5014b577443 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5fbfc7e4-416b-4694-8e99-2ed00a8df3b6 req-fec353e5-4ac5-4e10-a69e-3b9c7d8a38bf service nova] Acquiring lock "f1a0f3cf-1004-4ff9-9e41-d03291abbb9a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5fbfc7e4-416b-4694-8e99-2ed00a8df3b6 req-fec353e5-4ac5-4e10-a69e-3b9c7d8a38bf service nova] Lock "f1a0f3cf-1004-4ff9-9e41-d03291abbb9a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5fbfc7e4-416b-4694-8e99-2ed00a8df3b6 req-fec353e5-4ac5-4e10-a69e-3b9c7d8a38bf service nova] Lock "f1a0f3cf-1004-4ff9-9e41-d03291abbb9a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:22 user nova-compute[72081]: DEBUG nova.compute.manager [req-5fbfc7e4-416b-4694-8e99-2ed00a8df3b6 req-fec353e5-4ac5-4e10-a69e-3b9c7d8a38bf service nova] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] No waiting events found dispatching network-vif-plugged-e9d42f8f-a0eb-4058-83e3-f5014b577443 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:22 user nova-compute[72081]: WARNING nova.compute.manager [req-5fbfc7e4-416b-4694-8e99-2ed00a8df3b6 req-fec353e5-4ac5-4e10-a69e-3b9c7d8a38bf service nova] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Received unexpected event network-vif-plugged-e9d42f8f-a0eb-4058-83e3-f5014b577443 for instance with vm_state active and task_state None.
Apr 12 03:31:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-90d245db-ccfd-4b99-9938-f17c58f54d2c tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Lock "f1a0f3cf-1004-4ff9-9e41-d03291abbb9a" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 9.596s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:23 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Successfully updated port: 6bb0c89d-0a60-4413-b09d-08d4d6ea3a52 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:31:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:24 user nova-compute[72081]: DEBUG nova.compute.manager [req-0a1d6368-b29c-4a13-8935-b2a2a92153ec req-63e75760-012d-44ad-97aa-3ade75405076 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-changed-41626728-b21d-4ef8-9d86-5328d2feb3f9 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:24 user nova-compute[72081]: DEBUG nova.compute.manager [req-0a1d6368-b29c-4a13-8935-b2a2a92153ec req-63e75760-012d-44ad-97aa-3ade75405076 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Refreshing instance network info cache due to event network-changed-41626728-b21d-4ef8-9d86-5328d2feb3f9. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:31:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0a1d6368-b29c-4a13-8935-b2a2a92153ec req-63e75760-012d-44ad-97aa-3ade75405076 service nova] Acquiring lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:31:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0a1d6368-b29c-4a13-8935-b2a2a92153ec req-63e75760-012d-44ad-97aa-3ade75405076 service nova] Acquired lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:31:24 user nova-compute[72081]: DEBUG nova.network.neutron [req-0a1d6368-b29c-4a13-8935-b2a2a92153ec req-63e75760-012d-44ad-97aa-3ade75405076 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Refreshing network info cache for port 41626728-b21d-4ef8-9d86-5328d2feb3f9 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:31:24 user nova-compute[72081]: DEBUG nova.network.neutron [req-0a1d6368-b29c-4a13-8935-b2a2a92153ec req-63e75760-012d-44ad-97aa-3ade75405076 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:31:24 user nova-compute[72081]: DEBUG nova.network.neutron [req-0a1d6368-b29c-4a13-8935-b2a2a92153ec req-63e75760-012d-44ad-97aa-3ade75405076 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:31:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0a1d6368-b29c-4a13-8935-b2a2a92153ec req-63e75760-012d-44ad-97aa-3ade75405076 service nova] Releasing lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:31:24 user nova-compute[72081]: DEBUG nova.compute.manager [req-0a1d6368-b29c-4a13-8935-b2a2a92153ec req-63e75760-012d-44ad-97aa-3ade75405076 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-changed-6bb0c89d-0a60-4413-b09d-08d4d6ea3a52 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:24 user nova-compute[72081]: DEBUG nova.compute.manager [req-0a1d6368-b29c-4a13-8935-b2a2a92153ec req-63e75760-012d-44ad-97aa-3ade75405076 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Refreshing instance network info cache due to event network-changed-6bb0c89d-0a60-4413-b09d-08d4d6ea3a52. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:31:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0a1d6368-b29c-4a13-8935-b2a2a92153ec req-63e75760-012d-44ad-97aa-3ade75405076 service nova] Acquiring lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:31:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0a1d6368-b29c-4a13-8935-b2a2a92153ec req-63e75760-012d-44ad-97aa-3ade75405076 service nova] Acquired lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:31:24 user nova-compute[72081]: DEBUG nova.network.neutron [req-0a1d6368-b29c-4a13-8935-b2a2a92153ec req-63e75760-012d-44ad-97aa-3ade75405076 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Refreshing network info cache for port 6bb0c89d-0a60-4413-b09d-08d4d6ea3a52 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:31:24 user nova-compute[72081]: DEBUG nova.network.neutron [req-0a1d6368-b29c-4a13-8935-b2a2a92153ec req-63e75760-012d-44ad-97aa-3ade75405076 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Updating instance_info_cache with network_info: [{"id": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "address": "fa:16:3e:30:84:b9", "network": {"id": "9ea0ab38-8496-4a4b-a54b-fa36e90fd60a", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-849127753-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapce9193f1-28", "ovs_interfaceid": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "address": "fa:16:3e:56:d1:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.118", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap835b5a2a-4d", "ovs_interfaceid": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "a1aaff41-e246-4887-89c6-a91163d4efd5", "address": "fa:16:3e:d2:e8:92", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.42", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa1aaff41-e2", "ovs_interfaceid": "a1aaff41-e246-4887-89c6-a91163d4efd5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "ac401bfa-a662-4290-af6e-0a39e8fde61c", "address": "fa:16:3e:db:3e:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.161", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapac401bfa-a6", "ovs_interfaceid": "ac401bfa-a662-4290-af6e-0a39e8fde61c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "4e57de22-e876-4bfe-bd07-a1003afd2295", "address": "fa:16:3e:93:42:94", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4e57de22-e8", "ovs_interfaceid": "4e57de22-e876-4bfe-bd07-a1003afd2295", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8248913f-46c8-4f3d-ab47-0203cb267169", "address": "fa:16:3e:21:f0:77", "network": {"id": "435f9638-5e8f-40a8-b271-a183982c70d5", "bridge": "br-int", "label": "tempest-device-tagging-net2-1738895025", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8248913f-46", "ovs_interfaceid": "8248913f-46c8-4f3d-ab47-0203cb267169", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "21860860-2a08-43dc-bda8-0a1d0fab0634", "address": "fa:16:3e:eb:f1:cc", "network": {"id": "435f9638-5e8f-40a8-b271-a183982c70d5", "bridge": "br-int", "label": "tempest-device-tagging-net2-1738895025", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap21860860-2a", "ovs_interfaceid": "21860860-2a08-43dc-bda8-0a1d0fab0634", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Releasing lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Instance network_info: |[{"id": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "address": "fa:16:3e:30:84:b9", "network": {"id": "9ea0ab38-8496-4a4b-a54b-fa36e90fd60a", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-849127753-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapce9193f1-28", "ovs_interfaceid": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "address": "fa:16:3e:56:d1:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.118", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap835b5a2a-4d", "ovs_interfaceid": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "a1aaff41-e246-4887-89c6-a91163d4efd5", "address": "fa:16:3e:d2:e8:92", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.42", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa1aaff41-e2", "ovs_interfaceid": "a1aaff41-e246-4887-89c6-a91163d4efd5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "ac401bfa-a662-4290-af6e-0a39e8fde61c", "address": "fa:16:3e:db:3e:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.161", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapac401bfa-a6", "ovs_interfaceid": "ac401bfa-a662-4290-af6e-0a39e8fde61c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "4e57de22-e876-4bfe-bd07-a1003afd2295", "address": "fa:16:3e:93:42:94", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4e57de22-e8", "ovs_interfaceid": "4e57de22-e876-4bfe-bd07-a1003afd2295", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8248913f-46c8-4f3d-ab47-0203cb267169", "address": "fa:16:3e:21:f0:77", "network": {"id": "435f9638-5e8f-40a8-b271-a183982c70d5", "bridge": "br-int", "label": "tempest-device-tagging-net2-1738895025", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8248913f-46", "ovs_interfaceid": "8248913f-46c8-4f3d-ab47-0203cb267169", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "21860860-2a08-43dc-bda8-0a1d0fab0634", "address": "fa:16:3e:eb:f1:cc", "network": {"id": "435f9638-5e8f-40a8-b271-a183982c70d5", "bridge": "br-int", "label": "tempest-device-tagging-net2-1738895025", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap21860860-2a", "ovs_interfaceid": "21860860-2a08-43dc-bda8-0a1d0fab0634", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-de5a623d-7a92-4567-be13-724a403a9a5c req-167d98d8-9e72-460b-ba3b-4afb1f3742ab service nova] Acquired lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG nova.network.neutron [req-de5a623d-7a92-4567-be13-724a403a9a5c req-167d98d8-9e72-460b-ba3b-4afb1f3742ab service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Refreshing network info cache for port 21860860-2a08-43dc-bda8-0a1d0fab0634 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Start _get_guest_xml network_info=[{"id": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "address": "fa:16:3e:30:84:b9", "network": {"id": "9ea0ab38-8496-4a4b-a54b-fa36e90fd60a", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-849127753-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapce9193f1-28", "ovs_interfaceid": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "address": "fa:16:3e:56:d1:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.118", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap835b5a2a-4d", "ovs_interfaceid": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "a1aaff41-e246-4887-89c6-a91163d4efd5", "address": "fa:16:3e:d2:e8:92", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.42", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa1aaff41-e2", "ovs_interfaceid": "a1aaff41-e246-4887-89c6-a91163d4efd5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "ac401bfa-a662-4290-af6e-0a39e8fde61c", "address": "fa:16:3e:db:3e:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.161", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapac401bfa-a6", "ovs_interfaceid": "ac401bfa-a662-4290-af6e-0a39e8fde61c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "4e57de22-e876-4bfe-bd07-a1003afd2295", "address": "fa:16:3e:93:42:94", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4e57de22-e8", "ovs_interfaceid": "4e57de22-e876-4bfe-bd07-a1003afd2295", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8248913f-46c8-4f3d-ab47-0203cb267169", "address": "fa:16:3e:21:f0:77", "network": {"id": "435f9638-5e8f-40a8-b271-a183982c70d5", "bridge": "br-int", "label": "tempest-device-tagging-net2-1738895025", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8248913f-46", "ovs_interfaceid": "8248913f-46c8-4f3d-ab47-0203cb267169", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "21860860-2a08-43dc-bda8-0a1d0fab0634", "address": "fa:16:3e:eb:f1:cc", "network": {"id": "435f9638-5e8f-40a8-b271-a183982c70d5", "bridge": "br-int", "label": "tempest-device-tagging-net2-1738895025", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap21860860-2a", "ovs_interfaceid": "21860860-2a08-43dc-bda8-0a1d0fab0634", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk', 'boot_index': '2'}, '/dev/vdc': {'bus': 'virtio', 'dev': 'vdc', 'type': 'disk', 'boot_index': '3'}, 'disk.config': {'bus': 'ide', 'dev': 'hda', 'type': 'cdrom'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kdQHngfN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '7119b8b3-291b-4868-820e-d7bd3dd3ada4', 'attached_at': '', 'detached_at': '', 'volume_id': 'ba2eb790-6032-4455-82ed-af81c61ca352', 'serial': 'ba2eb790-6032-4455-82ed-af81c61ca352'}, 'disk_bus': 'virtio', 'delete_on_termination': False, 'boot_index': 0, 'guest_format': None, 'device_type': 'disk', 'attachment_id': 'a21851bc-89bb-4047-846a-b003155bc1d5', 'volume_type': None}, {'mount_device': '/dev/vdb', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'POhliZru', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '7119b8b3-291b-4868-820e-d7bd3dd3ada4', 'attached_at': '', 'detached_at': '', 'volume_id': '19d5feb6-1f29-4d61-8901-3b445f0c26fd', 'serial': '19d5feb6-1f29-4d61-8901-3b445f0c26fd'}, 'disk_bus': 'virtio', 'delete_on_termination': False, 'boot_index': 1, 'guest_format': None, 'device_type': 'disk', 'attachment_id': '9fa884b4-08e2-4abf-bb3d-64835659b883', 'volume_type': None}, {'mount_device': '/dev/vdc', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'PBUOnthu', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '7119b8b3-291b-4868-820e-d7bd3dd3ada4', 'attached_at': '', 'detached_at': '', 'volume_id': 'c73ac339-e533-4c8b-958d-ddf3e59dfb7d', 'serial': 'c73ac339-e533-4c8b-958d-ddf3e59dfb7d'}, 'disk_bus': 'virtio', 'delete_on_termination': False, 'boot_index': 2, 'guest_format': None, 'device_type': 'disk', 'attachment_id': '63c70663-4aef-4386-9dc5-ce7986e28ef0', 'volume_type': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:31:25 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:31:25 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:31:25 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Acquiring lock "cache_volume_driver" by "nova.virt.libvirt.driver.LibvirtDriver._get_volume_driver.<locals>._cache_volume_driver" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "cache_volume_driver" acquired by "nova.virt.libvirt.driver.LibvirtDriver._get_volume_driver.<locals>._cache_volume_driver" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG os_brick.initiator.connector [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Factory for ISCSI on None {{(pid=72081) factory /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connector.py:281}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "cache_volume_driver" "released" by "nova.virt.libvirt.driver.LibvirtDriver._get_volume_driver.<locals>._cache_volume_driver" :: held 0.008s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kdQHngfN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:31:25 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:31:25 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.003s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:31:25 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:31:25 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:31:25 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[74e2dd63-a817-46fe-9dbc-79e8c73252c8]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:31:25 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[43f73f3a-9a92-401e-8ec5-5ac72117db7a]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352] added
Apr 12 03:31:25 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ad0baba-f8e2-4fa1-b81a-a4e86439d7d6]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.034s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0c15b5fe-7b97-42c4-b58b-21aaadaf3900]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v kdQHngfN {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v kdQHngfN" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[29abb138-0690-4d83-94e4-1aa2a3225bce]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'kdQHngfN'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[91d0d7de-357a-42d5-b02f-188584d232fb]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2e86dc47-af0e-4677-a11c-ea0cd6d898cf]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Apr 12 03:31:25 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Apr 12 03:31:25 user nova-compute[72081]:  {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:31:25 user nova-compute[72081]: WARNING os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Apr 12 03:31:25 user nova-compute[72081]: 
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 -p 172.16.0.220:3260 --login" returned: 0 in 0.056s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1f482c2c-47ac-469c-8f4d-29e76a9a3bea]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352, portal: 172.16.0.220,3260]
Apr 12 03:31:25 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352, portal: 172.16.0.220,3260] successful.
Apr 12 03:31:25 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ff225ffe-420c-4b64-9b22-69c4b501db0f]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b174549e-725e-4acb-83bd-6ac3b705f0b3]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:31:25 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:31:25 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.298s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] HCTL ('33', '-', '-', 0) found on session 2 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[58a69fe5-f8bb-492c-85ce-c7b361979452]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Searching for a device in session 2 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kdQHngfN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Successfully updated port: e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Acquiring lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG nova.network.neutron [req-0a1d6368-b29c-4a13-8935-b2a2a92153ec req-63e75760-012d-44ad-97aa-3ade75405076 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0a1d6368-b29c-4a13-8935-b2a2a92153ec req-63e75760-012d-44ad-97aa-3ade75405076 service nova] Releasing lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Acquired lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:31:25 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-680158b1-d078-4407-bb04-2eb5f0a3eb02 req-caf43d50-3f35-43c4-bfc0-c303ca242ce9 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-changed-e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-680158b1-d078-4407-bb04-2eb5f0a3eb02 req-caf43d50-3f35-43c4-bfc0-c303ca242ce9 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Refreshing instance network info cache due to event network-changed-e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-680158b1-d078-4407-bb04-2eb5f0a3eb02 req-caf43d50-3f35-43c4-bfc0-c303ca242ce9 service nova] Acquiring lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG nova.network.neutron [req-de5a623d-7a92-4567-be13-724a403a9a5c req-167d98d8-9e72-460b-ba3b-4afb1f3742ab service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Updated VIF entry in instance network info cache for port 21860860-2a08-43dc-bda8-0a1d0fab0634. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG nova.network.neutron [req-de5a623d-7a92-4567-be13-724a403a9a5c req-167d98d8-9e72-460b-ba3b-4afb1f3742ab service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Updating instance_info_cache with network_info: [{"id": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "address": "fa:16:3e:30:84:b9", "network": {"id": "9ea0ab38-8496-4a4b-a54b-fa36e90fd60a", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-849127753-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapce9193f1-28", "ovs_interfaceid": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "address": "fa:16:3e:56:d1:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.118", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap835b5a2a-4d", "ovs_interfaceid": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "a1aaff41-e246-4887-89c6-a91163d4efd5", "address": "fa:16:3e:d2:e8:92", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.42", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa1aaff41-e2", "ovs_interfaceid": "a1aaff41-e246-4887-89c6-a91163d4efd5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "ac401bfa-a662-4290-af6e-0a39e8fde61c", "address": "fa:16:3e:db:3e:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.161", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapac401bfa-a6", "ovs_interfaceid": "ac401bfa-a662-4290-af6e-0a39e8fde61c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "4e57de22-e876-4bfe-bd07-a1003afd2295", "address": "fa:16:3e:93:42:94", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4e57de22-e8", "ovs_interfaceid": "4e57de22-e876-4bfe-bd07-a1003afd2295", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8248913f-46c8-4f3d-ab47-0203cb267169", "address": "fa:16:3e:21:f0:77", "network": {"id": "435f9638-5e8f-40a8-b271-a183982c70d5", "bridge": "br-int", "label": "tempest-device-tagging-net2-1738895025", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8248913f-46", "ovs_interfaceid": "8248913f-46c8-4f3d-ab47-0203cb267169", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "21860860-2a08-43dc-bda8-0a1d0fab0634", "address": "fa:16:3e:eb:f1:cc", "network": {"id": "435f9638-5e8f-40a8-b271-a183982c70d5", "bridge": "br-int", "label": "tempest-device-tagging-net2-1738895025", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap21860860-2a", "ovs_interfaceid": "21860860-2a08-43dc-bda8-0a1d0fab0634", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-de5a623d-7a92-4567-be13-724a403a9a5c req-167d98d8-9e72-460b-ba3b-4afb1f3742ab service nova] Releasing lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-de5a623d-7a92-4567-be13-724a403a9a5c req-167d98d8-9e72-460b-ba3b-4afb1f3742ab service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-changed-91177380-5d77-4bed-9ab0-e9085fcb3d5a {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-de5a623d-7a92-4567-be13-724a403a9a5c req-167d98d8-9e72-460b-ba3b-4afb1f3742ab service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Refreshing instance network info cache due to event network-changed-91177380-5d77-4bed-9ab0-e9085fcb3d5a. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-de5a623d-7a92-4567-be13-724a403a9a5c req-167d98d8-9e72-460b-ba3b-4afb1f3742ab service nova] Acquiring lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.337s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] <== connect_volume: return (1340ms) {'type': 'block', 'scsi_wwn': '26136333766636361', 'path': '/dev/sdb'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26136333766636361', 'path': '/dev/sdb'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'POhliZru', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:31:26 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:31:26 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:31:26 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:31:26 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:31:26 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd -p 172.16.0.220:3260" returned: 21 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[327f9de0-29a5-4097-a6be-e318c0fe2632]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:31:26 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[559c0d92-26f7-4583-a83b-066bf42abe1e]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd] added
Apr 12 03:31:26 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.034s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c7431f1-f1eb-461c-ac39-b6d82d632ff7]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.054s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[afbe1205-7b6a-48d1-a2e4-b590294cb2d2]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd -p 172.16.0.220:3260 --op update -n node.session.auth.username -v POhliZru {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd -p 172.16.0.220:3260 --op update -n node.session.auth.username -v POhliZru" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8dde1bfa-8ed5-4db5-9331-1f80b1bb4d66]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'POhliZru'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a9f3e60-7928-40c2-8e6c-63463d491739]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c5b50da1-11ac-400a-b32b-e4609685c5bd]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:31:26 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:31:26 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd -p 172.16.0.220:3260 --login" returned: 0 in 0.053s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[11954255-bcdd-4618-98ba-0ff42b9402fc]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd, portal: 172.16.0.220,3260]
Apr 12 03:31:26 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd, portal: 172.16.0.220,3260] successful.
Apr 12 03:31:26 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b4195d2b-e154-4e92-9eab-79a57f7d3a2e]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9bc322d8-9b25-4229-85b0-705d3ff6d843]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:31:26 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:31:26 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:31:26 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:31:26 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.331s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] HCTL ('34', '-', '-', 0) found on session 3 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ed74a370-a103-4940-a8f4-503ab99a20f0]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Searching for a device in session 3 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:31:26 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'POhliZru', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:31:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.375s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:31:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] <== connect_volume: return (1377ms) {'type': 'block', 'scsi_wwn': '26665393539373333', 'path': '/dev/sdc'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:31:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26665393539373333', 'path': '/dev/sdc'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG nova.network.neutron [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Updating instance_info_cache with network_info: [{"id": "33b2e7fb-520f-46d4-be86-e492aff77d4b", "address": "fa:16:3e:f1:16:6c", "network": {"id": "0905eabd-7263-448c-97f6-296c5b42cd24", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-192357488-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap33b2e7fb-52", "ovs_interfaceid": "33b2e7fb-520f-46d4-be86-e492aff77d4b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "7740f806-8319-484b-8b0f-bffd8c5fe532", "address": "fa:16:3e:10:f4:89", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.24", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7740f806-83", "ovs_interfaceid": "7740f806-8319-484b-8b0f-bffd8c5fe532", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "be3de447-6d22-4fec-b962-0ebe772ed86f", "address": "fa:16:3e:15:de:ed", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.143", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe3de447-6d", "ovs_interfaceid": "be3de447-6d22-4fec-b962-0ebe772ed86f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "address": "fa:16:3e:38:3d:5e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.54", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap91177380-5d", "ovs_interfaceid": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "address": "fa:16:3e:02:8f:9e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.17", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap41626728-b2", "ovs_interfaceid": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "address": "fa:16:3e:f2:df:e5", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bb0c89d-0a", "ovs_interfaceid": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "address": "fa:16:3e:db:9a:17", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16cfa52-5c", "ovs_interfaceid": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'PBUOnthu', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:31:28 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:31:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:31:28 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:31:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.004s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:31:28 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Releasing lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Instance network_info: |[{"id": "33b2e7fb-520f-46d4-be86-e492aff77d4b", "address": "fa:16:3e:f1:16:6c", "network": {"id": "0905eabd-7263-448c-97f6-296c5b42cd24", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-192357488-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap33b2e7fb-52", "ovs_interfaceid": "33b2e7fb-520f-46d4-be86-e492aff77d4b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "7740f806-8319-484b-8b0f-bffd8c5fe532", "address": "fa:16:3e:10:f4:89", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.24", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7740f806-83", "ovs_interfaceid": "7740f806-8319-484b-8b0f-bffd8c5fe532", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "be3de447-6d22-4fec-b962-0ebe772ed86f", "address": "fa:16:3e:15:de:ed", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.143", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe3de447-6d", "ovs_interfaceid": "be3de447-6d22-4fec-b962-0ebe772ed86f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "address": "fa:16:3e:38:3d:5e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.54", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap91177380-5d", "ovs_interfaceid": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "address": "fa:16:3e:02:8f:9e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.17", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap41626728-b2", "ovs_interfaceid": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "address": "fa:16:3e:f2:df:e5", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bb0c89d-0a", "ovs_interfaceid": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "address": "fa:16:3e:db:9a:17", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16cfa52-5c", "ovs_interfaceid": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-680158b1-d078-4407-bb04-2eb5f0a3eb02 req-caf43d50-3f35-43c4-bfc0-c303ca242ce9 service nova] Acquired lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG nova.network.neutron [req-680158b1-d078-4407-bb04-2eb5f0a3eb02 req-caf43d50-3f35-43c4-bfc0-c303ca242ce9 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Refreshing network info cache for port e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Start _get_guest_xml network_info=[{"id": "33b2e7fb-520f-46d4-be86-e492aff77d4b", "address": "fa:16:3e:f1:16:6c", "network": {"id": "0905eabd-7263-448c-97f6-296c5b42cd24", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-192357488-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap33b2e7fb-52", "ovs_interfaceid": "33b2e7fb-520f-46d4-be86-e492aff77d4b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "7740f806-8319-484b-8b0f-bffd8c5fe532", "address": "fa:16:3e:10:f4:89", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.24", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7740f806-83", "ovs_interfaceid": "7740f806-8319-484b-8b0f-bffd8c5fe532", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "be3de447-6d22-4fec-b962-0ebe772ed86f", "address": "fa:16:3e:15:de:ed", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.143", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe3de447-6d", "ovs_interfaceid": "be3de447-6d22-4fec-b962-0ebe772ed86f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "address": "fa:16:3e:38:3d:5e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.54", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap91177380-5d", "ovs_interfaceid": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "address": "fa:16:3e:02:8f:9e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.17", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap41626728-b2", "ovs_interfaceid": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "address": "fa:16:3e:f2:df:e5", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bb0c89d-0a", "ovs_interfaceid": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "address": "fa:16:3e:db:9a:17", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16cfa52-5c", "ovs_interfaceid": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk', 'boot_index': '2'}, '/dev/vdc': {'bus': 'virtio', 'dev': 'vdc', 'type': 'disk', 'boot_index': '3'}, 'disk.config': {'bus': 'ide', 'dev': 'hda', 'type': 'cdrom'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CmWAbLhi', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '43289ac5-3300-4e1f-8d07-2d66ee78e7ba', 'attached_at': '', 'detached_at': '', 'volume_id': '6a6c8492-c29c-49f6-954b-a9381cd886e1', 'serial': '6a6c8492-c29c-49f6-954b-a9381cd886e1'}, 'disk_bus': 'virtio', 'delete_on_termination': False, 'boot_index': 0, 'guest_format': None, 'device_type': 'disk', 'attachment_id': 'a9d33d1e-bb7e-42e3-a425-c102c25e85ba', 'volume_type': None}, {'mount_device': '/dev/vdb', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kNcbyUKL', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '43289ac5-3300-4e1f-8d07-2d66ee78e7ba', 'attached_at': '', 'detached_at': '', 'volume_id': '0aea0a9e-658d-4acc-ba07-3ef1bace5eb9', 'serial': '0aea0a9e-658d-4acc-ba07-3ef1bace5eb9'}, 'disk_bus': 'virtio', 'delete_on_termination': False, 'boot_index': 1, 'guest_format': None, 'device_type': 'disk', 'attachment_id': 'c671b848-20a3-459d-a7b2-d1ff1a072388', 'volume_type': None}, {'mount_device': '/dev/vdc', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'EUKqgyOv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '43289ac5-3300-4e1f-8d07-2d66ee78e7ba', 'attached_at': '', 'detached_at': '', 'volume_id': '6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3', 'serial': '6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3'}, 'disk_bus': 'virtio', 'delete_on_termination': False, 'boot_index': 2, 'guest_format': None, 'device_type': 'disk', 'attachment_id': '4a6652f8-39d1-49dc-992d-b402a6900247', 'volume_type': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0e9b099a-d2e5-4fd3-a083-58722b124f02]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:31:28 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:28 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:31:28 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:31:28 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CmWAbLhi', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:31:28 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:31:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[33d98784-8de3-4a77-b0c2-24ab6697f3a5]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d] added
Apr 12 03:31:28 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[658fcab4-2742-4eb5-9c50-9cf6579cccb8]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[188ff4ae-ebd5-4049-865d-e9fb00abbd1e]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d -p 172.16.0.220:3260 --op update -n node.session.auth.username -v PBUOnthu {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d -p 172.16.0.220:3260 --op update -n node.session.auth.username -v PBUOnthu" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[db15fa55-d1a1-473a-ba42-fdd524778c6b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'PBUOnthu'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bbf9bf26-17a6-472b-a8fa-6a255de6756d]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[de0ebde7-f45b-4a00-83e9-1841881e465f]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:31:28 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:31:28 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:31:28 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:31:28 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d -p 172.16.0.220:3260 --login" returned: 0 in 0.065s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2dc65aaf-8f46-416b-9ed9-3ee3672359b7]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d, portal: 172.16.0.220,3260]
Apr 12 03:31:28 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d, portal: 172.16.0.220,3260] successful.
Apr 12 03:31:28 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[61f81d2f-8f9b-4c38-b2cb-c2b184bc4a08]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[47546fe0-fe18-4e4d-9b33-ae9d2c83276b]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:31:28 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:31:28 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:31:28 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:31:28 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:31:28 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:31:28 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.327s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] HCTL ('35', '-', '-', 0) found on session 4 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8a1421fb-414a-4268-8cf2-a3935e34d3ed]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Searching for a device in session 4 and hctl ['35', '*', '*', 0] yield: None {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG nova.network.neutron [req-680158b1-d078-4407-bb04-2eb5f0a3eb02 req-caf43d50-3f35-43c4-bfc0-c303ca242ce9 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Updated VIF entry in instance network info cache for port e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG nova.network.neutron [req-680158b1-d078-4407-bb04-2eb5f0a3eb02 req-caf43d50-3f35-43c4-bfc0-c303ca242ce9 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Updating instance_info_cache with network_info: [{"id": "33b2e7fb-520f-46d4-be86-e492aff77d4b", "address": "fa:16:3e:f1:16:6c", "network": {"id": "0905eabd-7263-448c-97f6-296c5b42cd24", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-192357488-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap33b2e7fb-52", "ovs_interfaceid": "33b2e7fb-520f-46d4-be86-e492aff77d4b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "7740f806-8319-484b-8b0f-bffd8c5fe532", "address": "fa:16:3e:10:f4:89", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.24", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7740f806-83", "ovs_interfaceid": "7740f806-8319-484b-8b0f-bffd8c5fe532", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "be3de447-6d22-4fec-b962-0ebe772ed86f", "address": "fa:16:3e:15:de:ed", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.143", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe3de447-6d", "ovs_interfaceid": "be3de447-6d22-4fec-b962-0ebe772ed86f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "address": "fa:16:3e:38:3d:5e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.54", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap91177380-5d", "ovs_interfaceid": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "address": "fa:16:3e:02:8f:9e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.17", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap41626728-b2", "ovs_interfaceid": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "address": "fa:16:3e:f2:df:e5", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bb0c89d-0a", "ovs_interfaceid": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "address": "fa:16:3e:db:9a:17", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16cfa52-5c", "ovs_interfaceid": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-680158b1-d078-4407-bb04-2eb5f0a3eb02 req-caf43d50-3f35-43c4-bfc0-c303ca242ce9 service nova] Releasing lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-de5a623d-7a92-4567-be13-724a403a9a5c req-167d98d8-9e72-460b-ba3b-4afb1f3742ab service nova] Acquired lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG nova.network.neutron [req-de5a623d-7a92-4567-be13-724a403a9a5c req-167d98d8-9e72-460b-ba3b-4afb1f3742ab service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Refreshing network info cache for port 91177380-5d77-4bed-9ab0-e9085fcb3d5a {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:31:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.neutron [req-de5a623d-7a92-4567-be13-724a403a9a5c req-167d98d8-9e72-460b-ba3b-4afb1f3742ab service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Updated VIF entry in instance network info cache for port 91177380-5d77-4bed-9ab0-e9085fcb3d5a. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.neutron [req-de5a623d-7a92-4567-be13-724a403a9a5c req-167d98d8-9e72-460b-ba3b-4afb1f3742ab service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Updating instance_info_cache with network_info: [{"id": "33b2e7fb-520f-46d4-be86-e492aff77d4b", "address": "fa:16:3e:f1:16:6c", "network": {"id": "0905eabd-7263-448c-97f6-296c5b42cd24", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-192357488-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap33b2e7fb-52", "ovs_interfaceid": "33b2e7fb-520f-46d4-be86-e492aff77d4b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "7740f806-8319-484b-8b0f-bffd8c5fe532", "address": "fa:16:3e:10:f4:89", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.24", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7740f806-83", "ovs_interfaceid": "7740f806-8319-484b-8b0f-bffd8c5fe532", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "be3de447-6d22-4fec-b962-0ebe772ed86f", "address": "fa:16:3e:15:de:ed", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.143", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe3de447-6d", "ovs_interfaceid": "be3de447-6d22-4fec-b962-0ebe772ed86f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "address": "fa:16:3e:38:3d:5e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.54", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap91177380-5d", "ovs_interfaceid": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "address": "fa:16:3e:02:8f:9e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.17", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap41626728-b2", "ovs_interfaceid": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "address": "fa:16:3e:f2:df:e5", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bb0c89d-0a", "ovs_interfaceid": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "address": "fa:16:3e:db:9a:17", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16cfa52-5c", "ovs_interfaceid": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-de5a623d-7a92-4567-be13-724a403a9a5c req-167d98d8-9e72-460b-ba3b-4afb1f3742ab service nova] Releasing lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Searching for a device in session 4 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'PBUOnthu', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.360s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] <== connect_volume: return (1362ms) {'type': 'block', 'scsi_wwn': '26434353339333933', 'path': '/dev/sdd'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26434353339333933', 'path': '/dev/sdd'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 1.294s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:31:29 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:31:29 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:30:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1028113698',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1028113698',id=9,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS3ZbVHfxQVFHpZS5QTkoqKDiEjmhCdyP6+fgfZz0jwXQ4fAjcP60l9wC1gyScpF8Zw9rmHw2YJpeRpTUHuz2Hroka/B8qwVo0YxuHbLGdfKbP5YGwIEhUARrat7zqI4A==',key_name='tempest-keypair-476892975',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='72d31bafe182459aa8bcb4f1bbb5e6d0',ramdisk_id='',reservation_id='r-b6ok04uz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-294347777',owner_user_name='tempest-TaggedBootDevicesTest_v242-294347777-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e7aeea82653d4681aa22f6dd8ae742b4',uuid=7119b8b3-291b-4868-820e-d7bd3dd3ada4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "address": "fa:16:3e:30:84:b9", "network": {"id": "9ea0ab38-8496-4a4b-a54b-fa36e90fd60a", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-849127753-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapce9193f1-28", "ovs_interfaceid": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converting VIF {"id": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "address": "fa:16:3e:30:84:b9", "network": {"id": "9ea0ab38-8496-4a4b-a54b-fa36e90fd60a", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-849127753-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapce9193f1-28", "ovs_interfaceid": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:30:84:b9,bridge_name='br-int',has_traffic_filtering=True,id=ce9193f1-28a9-4c0f-95ae-e21f8140de19,network=Network(9ea0ab38-8496-4a4b-a54b-fa36e90fd60a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapce9193f1-28') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:30:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1028113698',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1028113698',id=9,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS3ZbVHfxQVFHpZS5QTkoqKDiEjmhCdyP6+fgfZz0jwXQ4fAjcP60l9wC1gyScpF8Zw9rmHw2YJpeRpTUHuz2Hroka/B8qwVo0YxuHbLGdfKbP5YGwIEhUARrat7zqI4A==',key_name='tempest-keypair-476892975',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='72d31bafe182459aa8bcb4f1bbb5e6d0',ramdisk_id='',reservation_id='r-b6ok04uz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-294347777',owner_user_name='tempest-TaggedBootDevicesTest_v242-294347777-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e7aeea82653d4681aa22f6dd8ae742b4',uuid=7119b8b3-291b-4868-820e-d7bd3dd3ada4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "address": "fa:16:3e:56:d1:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.118", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap835b5a2a-4d", "ovs_interfaceid": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converting VIF {"id": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "address": "fa:16:3e:56:d1:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.118", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap835b5a2a-4d", "ovs_interfaceid": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:56:d1:90,bridge_name='br-int',has_traffic_filtering=True,id=835b5a2a-4d35-4d2a-9fdd-0c78b7def163,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap835b5a2a-4d') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:30:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1028113698',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1028113698',id=9,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS3ZbVHfxQVFHpZS5QTkoqKDiEjmhCdyP6+fgfZz0jwXQ4fAjcP60l9wC1gyScpF8Zw9rmHw2YJpeRpTUHuz2Hroka/B8qwVo0YxuHbLGdfKbP5YGwIEhUARrat7zqI4A==',key_name='tempest-keypair-476892975',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='72d31bafe182459aa8bcb4f1bbb5e6d0',ramdisk_id='',reservation_id='r-b6ok04uz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-294347777',owner_user_name='tempest-TaggedBootDevicesTest_v242-294347777-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e7aeea82653d4681aa22f6dd8ae742b4',uuid=7119b8b3-291b-4868-820e-d7bd3dd3ada4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a1aaff41-e246-4887-89c6-a91163d4efd5", "address": "fa:16:3e:d2:e8:92", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.42", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa1aaff41-e2", "ovs_interfaceid": "a1aaff41-e246-4887-89c6-a91163d4efd5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converting VIF {"id": "a1aaff41-e246-4887-89c6-a91163d4efd5", "address": "fa:16:3e:d2:e8:92", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.42", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa1aaff41-e2", "ovs_interfaceid": "a1aaff41-e246-4887-89c6-a91163d4efd5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d2:e8:92,bridge_name='br-int',has_traffic_filtering=True,id=a1aaff41-e246-4887-89c6-a91163d4efd5,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapa1aaff41-e2') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:30:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1028113698',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1028113698',id=9,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS3ZbVHfxQVFHpZS5QTkoqKDiEjmhCdyP6+fgfZz0jwXQ4fAjcP60l9wC1gyScpF8Zw9rmHw2YJpeRpTUHuz2Hroka/B8qwVo0YxuHbLGdfKbP5YGwIEhUARrat7zqI4A==',key_name='tempest-keypair-476892975',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='72d31bafe182459aa8bcb4f1bbb5e6d0',ramdisk_id='',reservation_id='r-b6ok04uz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-294347777',owner_user_name='tempest-TaggedBootDevicesTest_v242-294347777-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e7aeea82653d4681aa22f6dd8ae742b4',uuid=7119b8b3-291b-4868-820e-d7bd3dd3ada4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "ac401bfa-a662-4290-af6e-0a39e8fde61c", "address": "fa:16:3e:db:3e:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.161", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapac401bfa-a6", "ovs_interfaceid": "ac401bfa-a662-4290-af6e-0a39e8fde61c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converting VIF {"id": "ac401bfa-a662-4290-af6e-0a39e8fde61c", "address": "fa:16:3e:db:3e:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.161", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapac401bfa-a6", "ovs_interfaceid": "ac401bfa-a662-4290-af6e-0a39e8fde61c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:db:3e:90,bridge_name='br-int',has_traffic_filtering=True,id=ac401bfa-a662-4290-af6e-0a39e8fde61c,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapac401bfa-a6') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:30:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1028113698',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1028113698',id=9,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS3ZbVHfxQVFHpZS5QTkoqKDiEjmhCdyP6+fgfZz0jwXQ4fAjcP60l9wC1gyScpF8Zw9rmHw2YJpeRpTUHuz2Hroka/B8qwVo0YxuHbLGdfKbP5YGwIEhUARrat7zqI4A==',key_name='tempest-keypair-476892975',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='72d31bafe182459aa8bcb4f1bbb5e6d0',ramdisk_id='',reservation_id='r-b6ok04uz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-294347777',owner_user_name='tempest-TaggedBootDevicesTest_v242-294347777-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e7aeea82653d4681aa22f6dd8ae742b4',uuid=7119b8b3-291b-4868-820e-d7bd3dd3ada4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "4e57de22-e876-4bfe-bd07-a1003afd2295", "address": "fa:16:3e:93:42:94", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4e57de22-e8", "ovs_interfaceid": "4e57de22-e876-4bfe-bd07-a1003afd2295", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converting VIF {"id": "4e57de22-e876-4bfe-bd07-a1003afd2295", "address": "fa:16:3e:93:42:94", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4e57de22-e8", "ovs_interfaceid": "4e57de22-e876-4bfe-bd07-a1003afd2295", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:93:42:94,bridge_name='br-int',has_traffic_filtering=True,id=4e57de22-e876-4bfe-bd07-a1003afd2295,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4e57de22-e8') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:30:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1028113698',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1028113698',id=9,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS3ZbVHfxQVFHpZS5QTkoqKDiEjmhCdyP6+fgfZz0jwXQ4fAjcP60l9wC1gyScpF8Zw9rmHw2YJpeRpTUHuz2Hroka/B8qwVo0YxuHbLGdfKbP5YGwIEhUARrat7zqI4A==',key_name='tempest-keypair-476892975',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='72d31bafe182459aa8bcb4f1bbb5e6d0',ramdisk_id='',reservation_id='r-b6ok04uz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-294347777',owner_user_name='tempest-TaggedBootDevicesTest_v242-294347777-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e7aeea82653d4681aa22f6dd8ae742b4',uuid=7119b8b3-291b-4868-820e-d7bd3dd3ada4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8248913f-46c8-4f3d-ab47-0203cb267169", "address": "fa:16:3e:21:f0:77", "network": {"id": "435f9638-5e8f-40a8-b271-a183982c70d5", "bridge": "br-int", "label": "tempest-device-tagging-net2-1738895025", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8248913f-46", "ovs_interfaceid": "8248913f-46c8-4f3d-ab47-0203cb267169", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converting VIF {"id": "8248913f-46c8-4f3d-ab47-0203cb267169", "address": "fa:16:3e:21:f0:77", "network": {"id": "435f9638-5e8f-40a8-b271-a183982c70d5", "bridge": "br-int", "label": "tempest-device-tagging-net2-1738895025", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8248913f-46", "ovs_interfaceid": "8248913f-46c8-4f3d-ab47-0203cb267169", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:21:f0:77,bridge_name='br-int',has_traffic_filtering=True,id=8248913f-46c8-4f3d-ab47-0203cb267169,network=Network(435f9638-5e8f-40a8-b271-a183982c70d5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8248913f-46') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:30:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1028113698',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1028113698',id=9,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS3ZbVHfxQVFHpZS5QTkoqKDiEjmhCdyP6+fgfZz0jwXQ4fAjcP60l9wC1gyScpF8Zw9rmHw2YJpeRpTUHuz2Hroka/B8qwVo0YxuHbLGdfKbP5YGwIEhUARrat7zqI4A==',key_name='tempest-keypair-476892975',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='72d31bafe182459aa8bcb4f1bbb5e6d0',ramdisk_id='',reservation_id='r-b6ok04uz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-294347777',owner_user_name='tempest-TaggedBootDevicesTest_v242-294347777-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e7aeea82653d4681aa22f6dd8ae742b4',uuid=7119b8b3-291b-4868-820e-d7bd3dd3ada4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "21860860-2a08-43dc-bda8-0a1d0fab0634", "address": "fa:16:3e:eb:f1:cc", "network": {"id": "435f9638-5e8f-40a8-b271-a183982c70d5", "bridge": "br-int", "label": "tempest-device-tagging-net2-1738895025", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap21860860-2a", "ovs_interfaceid": "21860860-2a08-43dc-bda8-0a1d0fab0634", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converting VIF {"id": "21860860-2a08-43dc-bda8-0a1d0fab0634", "address": "fa:16:3e:eb:f1:cc", "network": {"id": "435f9638-5e8f-40a8-b271-a183982c70d5", "bridge": "br-int", "label": "tempest-device-tagging-net2-1738895025", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap21860860-2a", "ovs_interfaceid": "21860860-2a08-43dc-bda8-0a1d0fab0634", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:eb:f1:cc,bridge_name='br-int',has_traffic_filtering=True,id=21860860-2a08-43dc-bda8-0a1d0fab0634,network=Network(435f9638-5e8f-40a8-b271-a183982c70d5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap21860860-2a') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.objects.instance [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lazy-loading 'pci_devices' on Instance uuid 7119b8b3-291b-4868-820e-d7bd3dd3ada4 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0413a0a3-6a57-47c9-a2f2-14dabca5c606]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:31:29 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:31:29 user nova-compute[72081]:   <uuid>7119b8b3-291b-4868-820e-d7bd3dd3ada4</uuid>
Apr 12 03:31:29 user nova-compute[72081]:   <name>instance-00000009</name>
Apr 12 03:31:29 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:31:29 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:31:29 user nova-compute[72081]:   <metadata>
Apr 12 03:31:29 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:31:29 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:31:29 user nova-compute[72081]:       <nova:name>tempest-device-tagging-server-1028113698</nova:name>
Apr 12 03:31:29 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:31:25</nova:creationTime>
Apr 12 03:31:29 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:31:29 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:31:29 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:31:29 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:31:29 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:31:29 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:31:29 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:31:29 user nova-compute[72081]:       <nova:owner>
Apr 12 03:31:29 user nova-compute[72081]:         <nova:user uuid="e7aeea82653d4681aa22f6dd8ae742b4">tempest-TaggedBootDevicesTest_v242-294347777-project-member</nova:user>
Apr 12 03:31:29 user nova-compute[72081]:         <nova:project uuid="72d31bafe182459aa8bcb4f1bbb5e6d0">tempest-TaggedBootDevicesTest_v242-294347777</nova:project>
Apr 12 03:31:29 user nova-compute[72081]:       </nova:owner>
Apr 12 03:31:29 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:31:29 user nova-compute[72081]:       <nova:ports>
Apr 12 03:31:29 user nova-compute[72081]:         <nova:port uuid="ce9193f1-28a9-4c0f-95ae-e21f8140de19">
Apr 12 03:31:29 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.8" ipVersion="4"/>
Apr 12 03:31:29 user nova-compute[72081]:         </nova:port>
Apr 12 03:31:29 user nova-compute[72081]:         <nova:port uuid="835b5a2a-4d35-4d2a-9fdd-0c78b7def163">
Apr 12 03:31:29 user nova-compute[72081]:           <nova:ip type="fixed" address="10.1.1.118" ipVersion="4"/>
Apr 12 03:31:29 user nova-compute[72081]:         </nova:port>
Apr 12 03:31:29 user nova-compute[72081]:         <nova:port uuid="a1aaff41-e246-4887-89c6-a91163d4efd5">
Apr 12 03:31:29 user nova-compute[72081]:           <nova:ip type="fixed" address="10.1.1.42" ipVersion="4"/>
Apr 12 03:31:29 user nova-compute[72081]:         </nova:port>
Apr 12 03:31:29 user nova-compute[72081]:         <nova:port uuid="ac401bfa-a662-4290-af6e-0a39e8fde61c">
Apr 12 03:31:29 user nova-compute[72081]:           <nova:ip type="fixed" address="10.1.1.161" ipVersion="4"/>
Apr 12 03:31:29 user nova-compute[72081]:         </nova:port>
Apr 12 03:31:29 user nova-compute[72081]:         <nova:port uuid="4e57de22-e876-4bfe-bd07-a1003afd2295">
Apr 12 03:31:29 user nova-compute[72081]:           <nova:ip type="fixed" address="10.1.1.149" ipVersion="4"/>
Apr 12 03:31:29 user nova-compute[72081]:         </nova:port>
Apr 12 03:31:29 user nova-compute[72081]:         <nova:port uuid="8248913f-46c8-4f3d-ab47-0203cb267169">
Apr 12 03:31:29 user nova-compute[72081]:           <nova:ip type="fixed" address="10.2.2.100" ipVersion="4"/>
Apr 12 03:31:29 user nova-compute[72081]:         </nova:port>
Apr 12 03:31:29 user nova-compute[72081]:         <nova:port uuid="21860860-2a08-43dc-bda8-0a1d0fab0634">
Apr 12 03:31:29 user nova-compute[72081]:           <nova:ip type="fixed" address="10.2.2.200" ipVersion="4"/>
Apr 12 03:31:29 user nova-compute[72081]:         </nova:port>
Apr 12 03:31:29 user nova-compute[72081]:       </nova:ports>
Apr 12 03:31:29 user nova-compute[72081]:     </nova:instance>
Apr 12 03:31:29 user nova-compute[72081]:   </metadata>
Apr 12 03:31:29 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:31:29 user nova-compute[72081]:     <system>
Apr 12 03:31:29 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:31:29 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:31:29 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:31:29 user nova-compute[72081]:       <entry name="serial">7119b8b3-291b-4868-820e-d7bd3dd3ada4</entry>
Apr 12 03:31:29 user nova-compute[72081]:       <entry name="uuid">7119b8b3-291b-4868-820e-d7bd3dd3ada4</entry>
Apr 12 03:31:29 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:31:29 user nova-compute[72081]:     </system>
Apr 12 03:31:29 user nova-compute[72081]:   </sysinfo>
Apr 12 03:31:29 user nova-compute[72081]:   <os>
Apr 12 03:31:29 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:31:29 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:31:29 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:31:29 user nova-compute[72081]:   </os>
Apr 12 03:31:29 user nova-compute[72081]:   <features>
Apr 12 03:31:29 user nova-compute[72081]:     <acpi/>
Apr 12 03:31:29 user nova-compute[72081]:     <apic/>
Apr 12 03:31:29 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:31:29 user nova-compute[72081]:   </features>
Apr 12 03:31:29 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:31:29 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:31:29 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:31:29 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:31:29 user nova-compute[72081]:   </clock>
Apr 12 03:31:29 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:31:29 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:31:29 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:31:29 user nova-compute[72081]:   </cpu>
Apr 12 03:31:29 user nova-compute[72081]:   <devices>
Apr 12 03:31:29 user nova-compute[72081]:     <disk type="file" device="cdrom">
Apr 12 03:31:29 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none"/>
Apr 12 03:31:29 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/7119b8b3-291b-4868-820e-d7bd3dd3ada4/disk.config"/>
Apr 12 03:31:29 user nova-compute[72081]:       <target dev="hda" bus="ide"/>
Apr 12 03:31:29 user nova-compute[72081]:     </disk>
Apr 12 03:31:29 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:31:29 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:31:29 user nova-compute[72081]:       <source dev="/dev/sdb"/>
Apr 12 03:31:29 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:31:29 user nova-compute[72081]:       <serial>ba2eb790-6032-4455-82ed-af81c61ca352</serial>
Apr 12 03:31:29 user nova-compute[72081]:     </disk>
Apr 12 03:31:29 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:31:29 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:31:29 user nova-compute[72081]:       <source dev="/dev/sdc"/>
Apr 12 03:31:29 user nova-compute[72081]:       <target dev="vdb" bus="virtio"/>
Apr 12 03:31:29 user nova-compute[72081]:       <serial>19d5feb6-1f29-4d61-8901-3b445f0c26fd</serial>
Apr 12 03:31:29 user nova-compute[72081]:     </disk>
Apr 12 03:31:29 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:31:29 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:31:29 user nova-compute[72081]:       <source dev="/dev/sdd"/>
Apr 12 03:31:29 user nova-compute[72081]:       <target dev="vdc" bus="virtio"/>
Apr 12 03:31:29 user nova-compute[72081]:       <serial>c73ac339-e533-4c8b-958d-ddf3e59dfb7d</serial>
Apr 12 03:31:29 user nova-compute[72081]:     </disk>
Apr 12 03:31:29 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:31:29 user nova-compute[72081]:       <mac address="fa:16:3e:30:84:b9"/>
Apr 12 03:31:29 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:31:29 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:31:29 user nova-compute[72081]:       <target dev="tapce9193f1-28"/>
Apr 12 03:31:29 user nova-compute[72081]:     </interface>
Apr 12 03:31:29 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:31:29 user nova-compute[72081]:       <mac address="fa:16:3e:56:d1:90"/>
Apr 12 03:31:29 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:31:29 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:31:29 user nova-compute[72081]:       <target dev="tap835b5a2a-4d"/>
Apr 12 03:31:29 user nova-compute[72081]:     </interface>
Apr 12 03:31:29 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:31:29 user nova-compute[72081]:       <mac address="fa:16:3e:d2:e8:92"/>
Apr 12 03:31:29 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:31:29 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:31:29 user nova-compute[72081]:       <target dev="tapa1aaff41-e2"/>
Apr 12 03:31:29 user nova-compute[72081]:     </interface>
Apr 12 03:31:29 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:31:29 user nova-compute[72081]:       <mac address="fa:16:3e:db:3e:90"/>
Apr 12 03:31:29 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:31:29 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:31:29 user nova-compute[72081]:       <target dev="tapac401bfa-a6"/>
Apr 12 03:31:29 user nova-compute[72081]:     </interface>
Apr 12 03:31:29 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:31:29 user nova-compute[72081]:       <mac address="fa:16:3e:93:42:94"/>
Apr 12 03:31:29 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:31:29 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:31:29 user nova-compute[72081]:       <target dev="tap4e57de22-e8"/>
Apr 12 03:31:29 user nova-compute[72081]:     </interface>
Apr 12 03:31:29 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:31:29 user nova-compute[72081]:       <mac address="fa:16:3e:21:f0:77"/>
Apr 12 03:31:29 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:31:29 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:31:29 user nova-compute[72081]:       <target dev="tap8248913f-46"/>
Apr 12 03:31:29 user nova-compute[72081]:     </interface>
Apr 12 03:31:29 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:31:29 user nova-compute[72081]:       <mac address="fa:16:3e:eb:f1:cc"/>
Apr 12 03:31:29 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:31:29 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:31:29 user nova-compute[72081]:       <target dev="tap21860860-2a"/>
Apr 12 03:31:29 user nova-compute[72081]:     </interface>
Apr 12 03:31:29 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:31:29 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/7119b8b3-291b-4868-820e-d7bd3dd3ada4/console.log" append="off"/>
Apr 12 03:31:29 user nova-compute[72081]:     </serial>
Apr 12 03:31:29 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:31:29 user nova-compute[72081]:     <video>
Apr 12 03:31:29 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:31:29 user nova-compute[72081]:     </video>
Apr 12 03:31:29 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:31:29 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:31:29 user nova-compute[72081]:     </rng>
Apr 12 03:31:29 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:31:29 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:31:29 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:31:29 user nova-compute[72081]:     </memballoon>
Apr 12 03:31:29 user nova-compute[72081]:   </devices>
Apr 12 03:31:29 user nova-compute[72081]: </domain>
Apr 12 03:31:29 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:30:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1028113698',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1028113698',id=9,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS3ZbVHfxQVFHpZS5QTkoqKDiEjmhCdyP6+fgfZz0jwXQ4fAjcP60l9wC1gyScpF8Zw9rmHw2YJpeRpTUHuz2Hroka/B8qwVo0YxuHbLGdfKbP5YGwIEhUARrat7zqI4A==',key_name='tempest-keypair-476892975',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='72d31bafe182459aa8bcb4f1bbb5e6d0',ramdisk_id='',reservation_id='r-b6ok04uz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-294347777',owner_user_name='tempest-TaggedBootDevicesTest_v242-294347777-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e7aeea82653d4681aa22f6dd8ae742b4',uuid=7119b8b3-291b-4868-820e-d7bd3dd3ada4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "address": "fa:16:3e:30:84:b9", "network": {"id": "9ea0ab38-8496-4a4b-a54b-fa36e90fd60a", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-849127753-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapce9193f1-28", "ovs_interfaceid": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converting VIF {"id": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "address": "fa:16:3e:30:84:b9", "network": {"id": "9ea0ab38-8496-4a4b-a54b-fa36e90fd60a", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-849127753-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapce9193f1-28", "ovs_interfaceid": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:30:84:b9,bridge_name='br-int',has_traffic_filtering=True,id=ce9193f1-28a9-4c0f-95ae-e21f8140de19,network=Network(9ea0ab38-8496-4a4b-a54b-fa36e90fd60a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapce9193f1-28') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:30:84:b9,bridge_name='br-int',has_traffic_filtering=True,id=ce9193f1-28a9-4c0f-95ae-e21f8140de19,network=Network(9ea0ab38-8496-4a4b-a54b-fa36e90fd60a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapce9193f1-28') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapce9193f1-28, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapce9193f1-28, col_values=(('external_ids', {'iface-id': 'ce9193f1-28a9-4c0f-95ae-e21f8140de19', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:30:84:b9', 'vm-uuid': '7119b8b3-291b-4868-820e-d7bd3dd3ada4'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[24fed59b-99c6-457f-ad84-fbc95ca97960]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1] added
Apr 12 03:31:29 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: INFO os_vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:30:84:b9,bridge_name='br-int',has_traffic_filtering=True,id=ce9193f1-28a9-4c0f-95ae-e21f8140de19,network=Network(9ea0ab38-8496-4a4b-a54b-fa36e90fd60a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapce9193f1-28')
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:30:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1028113698',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1028113698',id=9,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS3ZbVHfxQVFHpZS5QTkoqKDiEjmhCdyP6+fgfZz0jwXQ4fAjcP60l9wC1gyScpF8Zw9rmHw2YJpeRpTUHuz2Hroka/B8qwVo0YxuHbLGdfKbP5YGwIEhUARrat7zqI4A==',key_name='tempest-keypair-476892975',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='72d31bafe182459aa8bcb4f1bbb5e6d0',ramdisk_id='',reservation_id='r-b6ok04uz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-294347777',owner_user_name='tempest-TaggedBootDevicesTest_v242-294347777-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e7aeea82653d4681aa22f6dd8ae742b4',uuid=7119b8b3-291b-4868-820e-d7bd3dd3ada4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "address": "fa:16:3e:56:d1:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.118", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap835b5a2a-4d", "ovs_interfaceid": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converting VIF {"id": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "address": "fa:16:3e:56:d1:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.118", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap835b5a2a-4d", "ovs_interfaceid": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:56:d1:90,bridge_name='br-int',has_traffic_filtering=True,id=835b5a2a-4d35-4d2a-9fdd-0c78b7def163,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap835b5a2a-4d') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:56:d1:90,bridge_name='br-int',has_traffic_filtering=True,id=835b5a2a-4d35-4d2a-9fdd-0c78b7def163,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap835b5a2a-4d') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap835b5a2a-4d, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap835b5a2a-4d, col_values=(('external_ids', {'iface-id': '835b5a2a-4d35-4d2a-9fdd-0c78b7def163', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:56:d1:90', 'vm-uuid': '7119b8b3-291b-4868-820e-d7bd3dd3ada4'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9bfb6482-5c4a-4b43-83cf-beda124bd21b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: INFO os_vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:56:d1:90,bridge_name='br-int',has_traffic_filtering=True,id=835b5a2a-4d35-4d2a-9fdd-0c78b7def163,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap835b5a2a-4d')
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:30:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1028113698',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1028113698',id=9,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS3ZbVHfxQVFHpZS5QTkoqKDiEjmhCdyP6+fgfZz0jwXQ4fAjcP60l9wC1gyScpF8Zw9rmHw2YJpeRpTUHuz2Hroka/B8qwVo0YxuHbLGdfKbP5YGwIEhUARrat7zqI4A==',key_name='tempest-keypair-476892975',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='72d31bafe182459aa8bcb4f1bbb5e6d0',ramdisk_id='',reservation_id='r-b6ok04uz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-294347777',owner_user_name='tempest-TaggedBootDevicesTest_v242-294347777-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e7aeea82653d4681aa22f6dd8ae742b4',uuid=7119b8b3-291b-4868-820e-d7bd3dd3ada4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a1aaff41-e246-4887-89c6-a91163d4efd5", "address": "fa:16:3e:d2:e8:92", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.42", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa1aaff41-e2", "ovs_interfaceid": "a1aaff41-e246-4887-89c6-a91163d4efd5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converting VIF {"id": "a1aaff41-e246-4887-89c6-a91163d4efd5", "address": "fa:16:3e:d2:e8:92", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.42", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa1aaff41-e2", "ovs_interfaceid": "a1aaff41-e246-4887-89c6-a91163d4efd5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d2:e8:92,bridge_name='br-int',has_traffic_filtering=True,id=a1aaff41-e246-4887-89c6-a91163d4efd5,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapa1aaff41-e2') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:d2:e8:92,bridge_name='br-int',has_traffic_filtering=True,id=a1aaff41-e246-4887-89c6-a91163d4efd5,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapa1aaff41-e2') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapa1aaff41-e2, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapa1aaff41-e2, col_values=(('external_ids', {'iface-id': 'a1aaff41-e246-4887-89c6-a91163d4efd5', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:d2:e8:92', 'vm-uuid': '7119b8b3-291b-4868-820e-d7bd3dd3ada4'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4ca7d999-7cd9-4f1f-982c-2eef506b34e2]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v CmWAbLhi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: INFO os_vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:d2:e8:92,bridge_name='br-int',has_traffic_filtering=True,id=a1aaff41-e246-4887-89c6-a91163d4efd5,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapa1aaff41-e2')
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:30:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1028113698',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1028113698',id=9,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS3ZbVHfxQVFHpZS5QTkoqKDiEjmhCdyP6+fgfZz0jwXQ4fAjcP60l9wC1gyScpF8Zw9rmHw2YJpeRpTUHuz2Hroka/B8qwVo0YxuHbLGdfKbP5YGwIEhUARrat7zqI4A==',key_name='tempest-keypair-476892975',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='72d31bafe182459aa8bcb4f1bbb5e6d0',ramdisk_id='',reservation_id='r-b6ok04uz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-294347777',owner_user_name='tempest-TaggedBootDevicesTest_v242-294347777-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e7aeea82653d4681aa22f6dd8ae742b4',uuid=7119b8b3-291b-4868-820e-d7bd3dd3ada4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "ac401bfa-a662-4290-af6e-0a39e8fde61c", "address": "fa:16:3e:db:3e:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.161", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapac401bfa-a6", "ovs_interfaceid": "ac401bfa-a662-4290-af6e-0a39e8fde61c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converting VIF {"id": "ac401bfa-a662-4290-af6e-0a39e8fde61c", "address": "fa:16:3e:db:3e:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.161", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapac401bfa-a6", "ovs_interfaceid": "ac401bfa-a662-4290-af6e-0a39e8fde61c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:db:3e:90,bridge_name='br-int',has_traffic_filtering=True,id=ac401bfa-a662-4290-af6e-0a39e8fde61c,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapac401bfa-a6') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:db:3e:90,bridge_name='br-int',has_traffic_filtering=True,id=ac401bfa-a662-4290-af6e-0a39e8fde61c,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapac401bfa-a6') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v CmWAbLhi" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[137b73e1-30cd-4bd1-9bc7-2af10a5c7e69]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'CmWAbLhi'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapac401bfa-a6, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapac401bfa-a6, col_values=(('external_ids', {'iface-id': 'ac401bfa-a662-4290-af6e-0a39e8fde61c', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:db:3e:90', 'vm-uuid': '7119b8b3-291b-4868-820e-d7bd3dd3ada4'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7708f834-d48a-426e-b47b-3b62c137b0be]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: INFO os_vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:db:3e:90,bridge_name='br-int',has_traffic_filtering=True,id=ac401bfa-a662-4290-af6e-0a39e8fde61c,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapac401bfa-a6')
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:30:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1028113698',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1028113698',id=9,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS3ZbVHfxQVFHpZS5QTkoqKDiEjmhCdyP6+fgfZz0jwXQ4fAjcP60l9wC1gyScpF8Zw9rmHw2YJpeRpTUHuz2Hroka/B8qwVo0YxuHbLGdfKbP5YGwIEhUARrat7zqI4A==',key_name='tempest-keypair-476892975',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='72d31bafe182459aa8bcb4f1bbb5e6d0',ramdisk_id='',reservation_id='r-b6ok04uz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-294347777',owner_user_name='tempest-TaggedBootDevicesTest_v242-294347777-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e7aeea82653d4681aa22f6dd8ae742b4',uuid=7119b8b3-291b-4868-820e-d7bd3dd3ada4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "4e57de22-e876-4bfe-bd07-a1003afd2295", "address": "fa:16:3e:93:42:94", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4e57de22-e8", "ovs_interfaceid": "4e57de22-e876-4bfe-bd07-a1003afd2295", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converting VIF {"id": "4e57de22-e876-4bfe-bd07-a1003afd2295", "address": "fa:16:3e:93:42:94", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4e57de22-e8", "ovs_interfaceid": "4e57de22-e876-4bfe-bd07-a1003afd2295", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:93:42:94,bridge_name='br-int',has_traffic_filtering=True,id=4e57de22-e876-4bfe-bd07-a1003afd2295,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4e57de22-e8') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:93:42:94,bridge_name='br-int',has_traffic_filtering=True,id=4e57de22-e876-4bfe-bd07-a1003afd2295,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4e57de22-e8') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[aea3304b-2c3c-4361-b889-b0a4b7665a24]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:31:29 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:31:29 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:31:29 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:31:29 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:31:29 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:31:29 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap4e57de22-e8, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap4e57de22-e8, col_values=(('external_ids', {'iface-id': '4e57de22-e876-4bfe-bd07-a1003afd2295', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:93:42:94', 'vm-uuid': '7119b8b3-291b-4868-820e-d7bd3dd3ada4'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: INFO os_vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:93:42:94,bridge_name='br-int',has_traffic_filtering=True,id=4e57de22-e876-4bfe-bd07-a1003afd2295,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4e57de22-e8')
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:30:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1028113698',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1028113698',id=9,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS3ZbVHfxQVFHpZS5QTkoqKDiEjmhCdyP6+fgfZz0jwXQ4fAjcP60l9wC1gyScpF8Zw9rmHw2YJpeRpTUHuz2Hroka/B8qwVo0YxuHbLGdfKbP5YGwIEhUARrat7zqI4A==',key_name='tempest-keypair-476892975',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='72d31bafe182459aa8bcb4f1bbb5e6d0',ramdisk_id='',reservation_id='r-b6ok04uz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-294347777',owner_user_name='tempest-TaggedBootDevicesTest_v242-294347777-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e7aeea82653d4681aa22f6dd8ae742b4',uuid=7119b8b3-291b-4868-820e-d7bd3dd3ada4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8248913f-46c8-4f3d-ab47-0203cb267169", "address": "fa:16:3e:21:f0:77", "network": {"id": "435f9638-5e8f-40a8-b271-a183982c70d5", "bridge": "br-int", "label": "tempest-device-tagging-net2-1738895025", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8248913f-46", "ovs_interfaceid": "8248913f-46c8-4f3d-ab47-0203cb267169", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converting VIF {"id": "8248913f-46c8-4f3d-ab47-0203cb267169", "address": "fa:16:3e:21:f0:77", "network": {"id": "435f9638-5e8f-40a8-b271-a183982c70d5", "bridge": "br-int", "label": "tempest-device-tagging-net2-1738895025", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8248913f-46", "ovs_interfaceid": "8248913f-46c8-4f3d-ab47-0203cb267169", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:21:f0:77,bridge_name='br-int',has_traffic_filtering=True,id=8248913f-46c8-4f3d-ab47-0203cb267169,network=Network(435f9638-5e8f-40a8-b271-a183982c70d5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8248913f-46') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:21:f0:77,bridge_name='br-int',has_traffic_filtering=True,id=8248913f-46c8-4f3d-ab47-0203cb267169,network=Network(435f9638-5e8f-40a8-b271-a183982c70d5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8248913f-46') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap8248913f-46, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap8248913f-46, col_values=(('external_ids', {'iface-id': '8248913f-46c8-4f3d-ab47-0203cb267169', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:21:f0:77', 'vm-uuid': '7119b8b3-291b-4868-820e-d7bd3dd3ada4'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 -p 172.16.0.220:3260 --login" returned: 0 in 0.072s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e8b560d-c1ee-493e-b8b1-52dbccc58ca5]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1, portal: 172.16.0.220,3260]
Apr 12 03:31:29 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1, portal: 172.16.0.220,3260] successful.
Apr 12 03:31:29 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[07ac0c4a-aca5-4182-977e-43f5bdecb9bb]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b000ad4b-8203-4862-b50e-506f4ffb2a93]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:31:29 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:31:29 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:31:29 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:31:29 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:31:29 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:31:29 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:31:29 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:31:29 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.361s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] HCTL ('36', '-', '-', 0) found on session 5 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: INFO os_vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:21:f0:77,bridge_name='br-int',has_traffic_filtering=True,id=8248913f-46c8-4f3d-ab47-0203cb267169,network=Network(435f9638-5e8f-40a8-b271-a183982c70d5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8248913f-46')
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:30:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1028113698',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1028113698',id=9,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS3ZbVHfxQVFHpZS5QTkoqKDiEjmhCdyP6+fgfZz0jwXQ4fAjcP60l9wC1gyScpF8Zw9rmHw2YJpeRpTUHuz2Hroka/B8qwVo0YxuHbLGdfKbP5YGwIEhUARrat7zqI4A==',key_name='tempest-keypair-476892975',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='72d31bafe182459aa8bcb4f1bbb5e6d0',ramdisk_id='',reservation_id='r-b6ok04uz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-294347777',owner_user_name='tempest-TaggedBootDevicesTest_v242-294347777-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e7aeea82653d4681aa22f6dd8ae742b4',uuid=7119b8b3-291b-4868-820e-d7bd3dd3ada4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "21860860-2a08-43dc-bda8-0a1d0fab0634", "address": "fa:16:3e:eb:f1:cc", "network": {"id": "435f9638-5e8f-40a8-b271-a183982c70d5", "bridge": "br-int", "label": "tempest-device-tagging-net2-1738895025", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap21860860-2a", "ovs_interfaceid": "21860860-2a08-43dc-bda8-0a1d0fab0634", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converting VIF {"id": "21860860-2a08-43dc-bda8-0a1d0fab0634", "address": "fa:16:3e:eb:f1:cc", "network": {"id": "435f9638-5e8f-40a8-b271-a183982c70d5", "bridge": "br-int", "label": "tempest-device-tagging-net2-1738895025", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap21860860-2a", "ovs_interfaceid": "21860860-2a08-43dc-bda8-0a1d0fab0634", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:eb:f1:cc,bridge_name='br-int',has_traffic_filtering=True,id=21860860-2a08-43dc-bda8-0a1d0fab0634,network=Network(435f9638-5e8f-40a8-b271-a183982c70d5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap21860860-2a') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:eb:f1:cc,bridge_name='br-int',has_traffic_filtering=True,id=21860860-2a08-43dc-bda8-0a1d0fab0634,network=Network(435f9638-5e8f-40a8-b271-a183982c70d5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap21860860-2a') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap21860860-2a, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap21860860-2a, col_values=(('external_ids', {'iface-id': '21860860-2a08-43dc-bda8-0a1d0fab0634', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:eb:f1:cc', 'vm-uuid': '7119b8b3-291b-4868-820e-d7bd3dd3ada4'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[531cdeba-245b-42a5-bcb7-0ecc9ade693e]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Searching for a device in session 5 and hctl ['36', '*', '*', 0] yield: sde {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CmWAbLhi', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:31:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:29 user nova-compute[72081]: INFO os_vif [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:eb:f1:cc,bridge_name='br-int',has_traffic_filtering=True,id=21860860-2a08-43dc-bda8-0a1d0fab0634,network=Network(435f9638-5e8f-40a8-b271-a183982c70d5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap21860860-2a')
Apr 12 03:31:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] No BDM found with device name hda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:31:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] No BDM found with device name vdc, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:31:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] No VIF found with MAC fa:16:3e:30:84:b9, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:31:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] No VIF found with MAC fa:16:3e:93:42:94, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:31:30 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Using config drive
Apr 12 03:31:30 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Creating config drive at /opt/stack/data/nova/instances/7119b8b3-291b-4868-820e-d7bd3dd3ada4/disk.config
Apr 12 03:31:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Running cmd (subprocess): genisoimage -o /opt/stack/data/nova/instances/7119b8b3-291b-4868-820e-d7bd3dd3ada4/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpe2aqn5sc {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] CMD "genisoimage -o /opt/stack/data/nova/instances/7119b8b3-291b-4868-820e-d7bd3dd3ada4/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpe2aqn5sc" returned: 0 in 0.057s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.396s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:31:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] <== connect_volume: return (2690ms) {'type': 'block', 'scsi_wwn': '23161336636376331', 'path': '/dev/sde'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:31:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23161336636376331', 'path': '/dev/sde'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:31:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:31:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kNcbyUKL', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:31:30 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:31:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:31:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:31:30 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:31:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:31:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:31:30 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:31:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:30 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8a4f005c-5fc2-44d2-a5e8-fb163154726c]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:31:30 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:30 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b29137aa-95c8-4ff9-8d9e-1584051e1da1]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9] added
Apr 12 03:31:30 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:30 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5713e53a-de94-40d3-9bb9-119d21337ac4]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d20b6ad7-f17f-433f-b26e-98128e916239]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v kNcbyUKL {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v kNcbyUKL" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[80b0961b-f703-4392-ba9f-40456165fde5]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'kNcbyUKL'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a63da72e-9f9b-4b94-af2a-b285b38af4cb]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3199da7c-f918-4747-b0c3-45574b91ec1a]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:31:31 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:31:31 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:31:31 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:31:31 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:31:31 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:31:31 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:31:31 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:31:31 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 -p 172.16.0.220:3260 --login" returned: 0 in 0.071s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d4b6da8d-38e6-4216-bc83-03ab87c94759]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9, portal: 172.16.0.220,3260]
Apr 12 03:31:31 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9, portal: 172.16.0.220,3260] successful.
Apr 12 03:31:31 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[532ab85e-0b5b-40e7-a888-efb2729c0de6]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f1509bc5-4613-4669-b84f-d6617d7e7fcc]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:31:31 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:31:31 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:31:31 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:31:31 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:31:31 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:31:31 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:31:31 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:31:31 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:31:31 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:31:31 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.297s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] HCTL ('37', '-', '-', 0) found on session 6 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Scanning host 37 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d24da0ff-2830-4594-870e-a885da26af91]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Searching for a device in session 6 and hctl ['37', '*', '*', 0] yield: sdf {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Connected to sdf using {'target_iqn': 'iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kNcbyUKL', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG nova.compute.manager [req-87233480-805b-4014-8b03-4abef0513c57 req-fe1e671d-82f6-48c3-bc0c-d765cbb37fd7 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-plugged-ce9193f1-28a9-4c0f-95ae-e21f8140de19 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-87233480-805b-4014-8b03-4abef0513c57 req-fe1e671d-82f6-48c3-bc0c-d765cbb37fd7 service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-87233480-805b-4014-8b03-4abef0513c57 req-fe1e671d-82f6-48c3-bc0c-d765cbb37fd7 service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-87233480-805b-4014-8b03-4abef0513c57 req-fe1e671d-82f6-48c3-bc0c-d765cbb37fd7 service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:31 user nova-compute[72081]: DEBUG nova.compute.manager [req-87233480-805b-4014-8b03-4abef0513c57 req-fe1e671d-82f6-48c3-bc0c-d765cbb37fd7 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-plugged-ce9193f1-28a9-4c0f-95ae-e21f8140de19 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:31 user nova-compute[72081]: WARNING nova.compute.manager [req-87233480-805b-4014-8b03-4abef0513c57 req-fe1e671d-82f6-48c3-bc0c-d765cbb37fd7 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received unexpected event network-vif-plugged-ce9193f1-28a9-4c0f-95ae-e21f8140de19 for instance with vm_state building and task_state spawning.
Apr 12 03:31:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.332s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] <== connect_volume: return (1335ms) {'type': 'block', 'scsi_wwn': '23432616236653331', 'path': '/dev/sdf'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23432616236653331', 'path': '/dev/sdf'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'EUKqgyOv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:31:32 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:31:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:31:32 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:31:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:31:32 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 -p 172.16.0.220:3260" returned: 21 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0046af56-70a6-45fe-a1e5-7ea313f2a16a]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:31:32 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[80e5d650-d01d-401d-b91a-6f861b109f35]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3] added
Apr 12 03:31:32 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.033s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4fee85df-bfb5-4aa0-b484-d979b5ef3fc9]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[152c4baa-1a0f-43a0-89d2-2bd806412b54]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v EUKqgyOv {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v EUKqgyOv" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[26a08437-9acb-4fda-8e05-81651c65b83f]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'EUKqgyOv'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[97771653-1ccc-4c32-ab3f-2fde37a0f59a]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.034s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c90cb0f1-2c18-4db4-b2b4-91e0083fb6a7]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:31:32 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:31:32 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:31:32 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:31:32 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:31:32 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:31:32 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:31:32 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:31:32 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:31:32 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:31:32 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 -p 172.16.0.220:3260 --login" returned: 0 in 0.100s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2f308738-b32c-4e6a-9436-9784acbd6717]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3, portal: 172.16.0.220,3260]
Apr 12 03:31:32 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3, portal: 172.16.0.220,3260] successful.
Apr 12 03:31:32 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[609975f8-0e35-441b-bc70-29385adff6c1]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.037s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e0953f16-0dff-4ede-9e69-7f379e73bf92]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:31:32 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:31:32 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:31:32 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:31:32 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:31:32 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:31:32 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:31:32 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:31:32 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:31:32 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:31:32 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:31:32 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:31:32 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.395s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] HCTL ('38', '-', '-', 0) found on session 7 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Scanning host 38 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host38/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host38/scan" returned: 0 in 0.037s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[68555ed3-3776-4e02-933b-43d396d41aff]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Searching for a device in session 7 and hctl ['38', '*', '*', 0] yield: None {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG nova.compute.manager [req-a2549a14-b550-4e4e-bc5a-3ff351476a91 req-6b227506-70c8-44e5-a89c-fb9e02f29e34 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-plugged-ac401bfa-a662-4290-af6e-0a39e8fde61c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a2549a14-b550-4e4e-bc5a-3ff351476a91 req-6b227506-70c8-44e5-a89c-fb9e02f29e34 service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a2549a14-b550-4e4e-bc5a-3ff351476a91 req-6b227506-70c8-44e5-a89c-fb9e02f29e34 service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a2549a14-b550-4e4e-bc5a-3ff351476a91 req-6b227506-70c8-44e5-a89c-fb9e02f29e34 service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG nova.compute.manager [req-a2549a14-b550-4e4e-bc5a-3ff351476a91 req-6b227506-70c8-44e5-a89c-fb9e02f29e34 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-plugged-ac401bfa-a662-4290-af6e-0a39e8fde61c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:32 user nova-compute[72081]: WARNING nova.compute.manager [req-a2549a14-b550-4e4e-bc5a-3ff351476a91 req-6b227506-70c8-44e5-a89c-fb9e02f29e34 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received unexpected event network-vif-plugged-ac401bfa-a662-4290-af6e-0a39e8fde61c for instance with vm_state building and task_state spawning.
Apr 12 03:31:32 user nova-compute[72081]: DEBUG nova.compute.manager [req-3a034232-f78e-4323-9c73-270fd9a46bae req-849e7f74-e6f2-4d74-9818-7af26cceafd7 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-plugged-a1aaff41-e246-4887-89c6-a91163d4efd5 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3a034232-f78e-4323-9c73-270fd9a46bae req-849e7f74-e6f2-4d74-9818-7af26cceafd7 service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3a034232-f78e-4323-9c73-270fd9a46bae req-849e7f74-e6f2-4d74-9818-7af26cceafd7 service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3a034232-f78e-4323-9c73-270fd9a46bae req-849e7f74-e6f2-4d74-9818-7af26cceafd7 service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG nova.compute.manager [req-3a034232-f78e-4323-9c73-270fd9a46bae req-849e7f74-e6f2-4d74-9818-7af26cceafd7 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-plugged-a1aaff41-e246-4887-89c6-a91163d4efd5 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:32 user nova-compute[72081]: WARNING nova.compute.manager [req-3a034232-f78e-4323-9c73-270fd9a46bae req-849e7f74-e6f2-4d74-9818-7af26cceafd7 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received unexpected event network-vif-plugged-a1aaff41-e246-4887-89c6-a91163d4efd5 for instance with vm_state building and task_state spawning.
Apr 12 03:31:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Searching for a device in session 7 and hctl ['38', '*', '*', 0] yield: sdg {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Connected to sdg using {'target_iqn': 'iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'EUKqgyOv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.445s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] <== connect_volume: return (1446ms) {'type': 'block', 'scsi_wwn': '23133666236636464', 'path': '/dev/sdg'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23133666236636464', 'path': '/dev/sdg'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:31:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-780207909',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-780207909',id=10,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNCyWVMCVeXTj4eZLmBDk3uEr+ohaWpwIv6zTkIlNZ2QRH2THWN8SXei5drq2IJGQkIvQoNzOXVCuGCaf7h5mScjI1xfzmSUsrhlqHzEQ4MwAG4OF6CGlMBmuH+1KctDlA==',key_name='tempest-keypair-2115967448',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bd808e7e237f4c7490abaff5db0e3529',ramdisk_id='',reservation_id='r-vmalcql5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1194851633',owner_user_name='tempest-TaggedBootDevicesTest-1194851633-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:04Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d29b6ed431d64bcab02c2348feab9aa6',uuid=43289ac5-3300-4e1f-8d07-2d66ee78e7ba,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "33b2e7fb-520f-46d4-be86-e492aff77d4b", "address": "fa:16:3e:f1:16:6c", "network": {"id": "0905eabd-7263-448c-97f6-296c5b42cd24", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-192357488-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap33b2e7fb-52", "ovs_interfaceid": "33b2e7fb-520f-46d4-be86-e492aff77d4b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converting VIF {"id": "33b2e7fb-520f-46d4-be86-e492aff77d4b", "address": "fa:16:3e:f1:16:6c", "network": {"id": "0905eabd-7263-448c-97f6-296c5b42cd24", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-192357488-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap33b2e7fb-52", "ovs_interfaceid": "33b2e7fb-520f-46d4-be86-e492aff77d4b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f1:16:6c,bridge_name='br-int',has_traffic_filtering=True,id=33b2e7fb-520f-46d4-be86-e492aff77d4b,network=Network(0905eabd-7263-448c-97f6-296c5b42cd24),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap33b2e7fb-52') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:31:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-780207909',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-780207909',id=10,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNCyWVMCVeXTj4eZLmBDk3uEr+ohaWpwIv6zTkIlNZ2QRH2THWN8SXei5drq2IJGQkIvQoNzOXVCuGCaf7h5mScjI1xfzmSUsrhlqHzEQ4MwAG4OF6CGlMBmuH+1KctDlA==',key_name='tempest-keypair-2115967448',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bd808e7e237f4c7490abaff5db0e3529',ramdisk_id='',reservation_id='r-vmalcql5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1194851633',owner_user_name='tempest-TaggedBootDevicesTest-1194851633-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:04Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d29b6ed431d64bcab02c2348feab9aa6',uuid=43289ac5-3300-4e1f-8d07-2d66ee78e7ba,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7740f806-8319-484b-8b0f-bffd8c5fe532", "address": "fa:16:3e:10:f4:89", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.24", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7740f806-83", "ovs_interfaceid": "7740f806-8319-484b-8b0f-bffd8c5fe532", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converting VIF {"id": "7740f806-8319-484b-8b0f-bffd8c5fe532", "address": "fa:16:3e:10:f4:89", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.24", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7740f806-83", "ovs_interfaceid": "7740f806-8319-484b-8b0f-bffd8c5fe532", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:10:f4:89,bridge_name='br-int',has_traffic_filtering=True,id=7740f806-8319-484b-8b0f-bffd8c5fe532,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap7740f806-83') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:31:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-780207909',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-780207909',id=10,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNCyWVMCVeXTj4eZLmBDk3uEr+ohaWpwIv6zTkIlNZ2QRH2THWN8SXei5drq2IJGQkIvQoNzOXVCuGCaf7h5mScjI1xfzmSUsrhlqHzEQ4MwAG4OF6CGlMBmuH+1KctDlA==',key_name='tempest-keypair-2115967448',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bd808e7e237f4c7490abaff5db0e3529',ramdisk_id='',reservation_id='r-vmalcql5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1194851633',owner_user_name='tempest-TaggedBootDevicesTest-1194851633-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:04Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d29b6ed431d64bcab02c2348feab9aa6',uuid=43289ac5-3300-4e1f-8d07-2d66ee78e7ba,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "be3de447-6d22-4fec-b962-0ebe772ed86f", "address": "fa:16:3e:15:de:ed", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.143", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe3de447-6d", "ovs_interfaceid": "be3de447-6d22-4fec-b962-0ebe772ed86f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converting VIF {"id": "be3de447-6d22-4fec-b962-0ebe772ed86f", "address": "fa:16:3e:15:de:ed", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.143", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe3de447-6d", "ovs_interfaceid": "be3de447-6d22-4fec-b962-0ebe772ed86f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:15:de:ed,bridge_name='br-int',has_traffic_filtering=True,id=be3de447-6d22-4fec-b962-0ebe772ed86f,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapbe3de447-6d') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:31:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-780207909',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-780207909',id=10,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNCyWVMCVeXTj4eZLmBDk3uEr+ohaWpwIv6zTkIlNZ2QRH2THWN8SXei5drq2IJGQkIvQoNzOXVCuGCaf7h5mScjI1xfzmSUsrhlqHzEQ4MwAG4OF6CGlMBmuH+1KctDlA==',key_name='tempest-keypair-2115967448',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bd808e7e237f4c7490abaff5db0e3529',ramdisk_id='',reservation_id='r-vmalcql5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1194851633',owner_user_name='tempest-TaggedBootDevicesTest-1194851633-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:04Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d29b6ed431d64bcab02c2348feab9aa6',uuid=43289ac5-3300-4e1f-8d07-2d66ee78e7ba,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "address": "fa:16:3e:38:3d:5e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.54", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap91177380-5d", "ovs_interfaceid": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converting VIF {"id": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "address": "fa:16:3e:38:3d:5e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.54", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap91177380-5d", "ovs_interfaceid": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:38:3d:5e,bridge_name='br-int',has_traffic_filtering=True,id=91177380-5d77-4bed-9ab0-e9085fcb3d5a,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap91177380-5d') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:31:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-780207909',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-780207909',id=10,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNCyWVMCVeXTj4eZLmBDk3uEr+ohaWpwIv6zTkIlNZ2QRH2THWN8SXei5drq2IJGQkIvQoNzOXVCuGCaf7h5mScjI1xfzmSUsrhlqHzEQ4MwAG4OF6CGlMBmuH+1KctDlA==',key_name='tempest-keypair-2115967448',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bd808e7e237f4c7490abaff5db0e3529',ramdisk_id='',reservation_id='r-vmalcql5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1194851633',owner_user_name='tempest-TaggedBootDevicesTest-1194851633-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:04Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d29b6ed431d64bcab02c2348feab9aa6',uuid=43289ac5-3300-4e1f-8d07-2d66ee78e7ba,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "address": "fa:16:3e:02:8f:9e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.17", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap41626728-b2", "ovs_interfaceid": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converting VIF {"id": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "address": "fa:16:3e:02:8f:9e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.17", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap41626728-b2", "ovs_interfaceid": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:02:8f:9e,bridge_name='br-int',has_traffic_filtering=True,id=41626728-b21d-4ef8-9d86-5328d2feb3f9,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap41626728-b2') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:31:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-780207909',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-780207909',id=10,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNCyWVMCVeXTj4eZLmBDk3uEr+ohaWpwIv6zTkIlNZ2QRH2THWN8SXei5drq2IJGQkIvQoNzOXVCuGCaf7h5mScjI1xfzmSUsrhlqHzEQ4MwAG4OF6CGlMBmuH+1KctDlA==',key_name='tempest-keypair-2115967448',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bd808e7e237f4c7490abaff5db0e3529',ramdisk_id='',reservation_id='r-vmalcql5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1194851633',owner_user_name='tempest-TaggedBootDevicesTest-1194851633-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:04Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d29b6ed431d64bcab02c2348feab9aa6',uuid=43289ac5-3300-4e1f-8d07-2d66ee78e7ba,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "address": "fa:16:3e:f2:df:e5", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bb0c89d-0a", "ovs_interfaceid": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converting VIF {"id": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "address": "fa:16:3e:f2:df:e5", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bb0c89d-0a", "ovs_interfaceid": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f2:df:e5,bridge_name='br-int',has_traffic_filtering=True,id=6bb0c89d-0a60-4413-b09d-08d4d6ea3a52,network=Network(2767acc3-5e96-41ad-a7f4-fbd5b4acce3a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6bb0c89d-0a') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:31:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-780207909',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-780207909',id=10,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNCyWVMCVeXTj4eZLmBDk3uEr+ohaWpwIv6zTkIlNZ2QRH2THWN8SXei5drq2IJGQkIvQoNzOXVCuGCaf7h5mScjI1xfzmSUsrhlqHzEQ4MwAG4OF6CGlMBmuH+1KctDlA==',key_name='tempest-keypair-2115967448',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bd808e7e237f4c7490abaff5db0e3529',ramdisk_id='',reservation_id='r-vmalcql5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1194851633',owner_user_name='tempest-TaggedBootDevicesTest-1194851633-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:04Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d29b6ed431d64bcab02c2348feab9aa6',uuid=43289ac5-3300-4e1f-8d07-2d66ee78e7ba,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "address": "fa:16:3e:db:9a:17", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16cfa52-5c", "ovs_interfaceid": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converting VIF {"id": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "address": "fa:16:3e:db:9a:17", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16cfa52-5c", "ovs_interfaceid": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:db:9a:17,bridge_name='br-int',has_traffic_filtering=True,id=e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1,network=Network(2767acc3-5e96-41ad-a7f4-fbd5b4acce3a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16cfa52-5c') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.objects.instance [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lazy-loading 'pci_devices' on Instance uuid 43289ac5-3300-4e1f-8d07-2d66ee78e7ba {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:31:33 user nova-compute[72081]:   <uuid>43289ac5-3300-4e1f-8d07-2d66ee78e7ba</uuid>
Apr 12 03:31:33 user nova-compute[72081]:   <name>instance-0000000a</name>
Apr 12 03:31:33 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:31:33 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:31:33 user nova-compute[72081]:   <metadata>
Apr 12 03:31:33 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:31:33 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:31:33 user nova-compute[72081]:       <nova:name>tempest-device-tagging-server-780207909</nova:name>
Apr 12 03:31:33 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:31:28</nova:creationTime>
Apr 12 03:31:33 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:31:33 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:31:33 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:31:33 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:31:33 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:31:33 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:31:33 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:31:33 user nova-compute[72081]:       <nova:owner>
Apr 12 03:31:33 user nova-compute[72081]:         <nova:user uuid="d29b6ed431d64bcab02c2348feab9aa6">tempest-TaggedBootDevicesTest-1194851633-project-member</nova:user>
Apr 12 03:31:33 user nova-compute[72081]:         <nova:project uuid="bd808e7e237f4c7490abaff5db0e3529">tempest-TaggedBootDevicesTest-1194851633</nova:project>
Apr 12 03:31:33 user nova-compute[72081]:       </nova:owner>
Apr 12 03:31:33 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:31:33 user nova-compute[72081]:       <nova:ports>
Apr 12 03:31:33 user nova-compute[72081]:         <nova:port uuid="33b2e7fb-520f-46d4-be86-e492aff77d4b">
Apr 12 03:31:33 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Apr 12 03:31:33 user nova-compute[72081]:         </nova:port>
Apr 12 03:31:33 user nova-compute[72081]:         <nova:port uuid="7740f806-8319-484b-8b0f-bffd8c5fe532">
Apr 12 03:31:33 user nova-compute[72081]:           <nova:ip type="fixed" address="10.1.1.24" ipVersion="4"/>
Apr 12 03:31:33 user nova-compute[72081]:         </nova:port>
Apr 12 03:31:33 user nova-compute[72081]:         <nova:port uuid="be3de447-6d22-4fec-b962-0ebe772ed86f">
Apr 12 03:31:33 user nova-compute[72081]:           <nova:ip type="fixed" address="10.1.1.143" ipVersion="4"/>
Apr 12 03:31:33 user nova-compute[72081]:         </nova:port>
Apr 12 03:31:33 user nova-compute[72081]:         <nova:port uuid="91177380-5d77-4bed-9ab0-e9085fcb3d5a">
Apr 12 03:31:33 user nova-compute[72081]:           <nova:ip type="fixed" address="10.1.1.54" ipVersion="4"/>
Apr 12 03:31:33 user nova-compute[72081]:         </nova:port>
Apr 12 03:31:33 user nova-compute[72081]:         <nova:port uuid="41626728-b21d-4ef8-9d86-5328d2feb3f9">
Apr 12 03:31:33 user nova-compute[72081]:           <nova:ip type="fixed" address="10.1.1.17" ipVersion="4"/>
Apr 12 03:31:33 user nova-compute[72081]:         </nova:port>
Apr 12 03:31:33 user nova-compute[72081]:         <nova:port uuid="6bb0c89d-0a60-4413-b09d-08d4d6ea3a52">
Apr 12 03:31:33 user nova-compute[72081]:           <nova:ip type="fixed" address="10.2.2.100" ipVersion="4"/>
Apr 12 03:31:33 user nova-compute[72081]:         </nova:port>
Apr 12 03:31:33 user nova-compute[72081]:         <nova:port uuid="e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1">
Apr 12 03:31:33 user nova-compute[72081]:           <nova:ip type="fixed" address="10.2.2.200" ipVersion="4"/>
Apr 12 03:31:33 user nova-compute[72081]:         </nova:port>
Apr 12 03:31:33 user nova-compute[72081]:       </nova:ports>
Apr 12 03:31:33 user nova-compute[72081]:     </nova:instance>
Apr 12 03:31:33 user nova-compute[72081]:   </metadata>
Apr 12 03:31:33 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:31:33 user nova-compute[72081]:     <system>
Apr 12 03:31:33 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:31:33 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:31:33 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:31:33 user nova-compute[72081]:       <entry name="serial">43289ac5-3300-4e1f-8d07-2d66ee78e7ba</entry>
Apr 12 03:31:33 user nova-compute[72081]:       <entry name="uuid">43289ac5-3300-4e1f-8d07-2d66ee78e7ba</entry>
Apr 12 03:31:33 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:31:33 user nova-compute[72081]:     </system>
Apr 12 03:31:33 user nova-compute[72081]:   </sysinfo>
Apr 12 03:31:33 user nova-compute[72081]:   <os>
Apr 12 03:31:33 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:31:33 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:31:33 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:31:33 user nova-compute[72081]:   </os>
Apr 12 03:31:33 user nova-compute[72081]:   <features>
Apr 12 03:31:33 user nova-compute[72081]:     <acpi/>
Apr 12 03:31:33 user nova-compute[72081]:     <apic/>
Apr 12 03:31:33 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:31:33 user nova-compute[72081]:   </features>
Apr 12 03:31:33 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:31:33 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:31:33 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:31:33 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:31:33 user nova-compute[72081]:   </clock>
Apr 12 03:31:33 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:31:33 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:31:33 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:31:33 user nova-compute[72081]:   </cpu>
Apr 12 03:31:33 user nova-compute[72081]:   <devices>
Apr 12 03:31:33 user nova-compute[72081]:     <disk type="file" device="cdrom">
Apr 12 03:31:33 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none"/>
Apr 12 03:31:33 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/43289ac5-3300-4e1f-8d07-2d66ee78e7ba/disk.config"/>
Apr 12 03:31:33 user nova-compute[72081]:       <target dev="hda" bus="ide"/>
Apr 12 03:31:33 user nova-compute[72081]:     </disk>
Apr 12 03:31:33 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:31:33 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:31:33 user nova-compute[72081]:       <source dev="/dev/sde"/>
Apr 12 03:31:33 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:31:33 user nova-compute[72081]:       <serial>6a6c8492-c29c-49f6-954b-a9381cd886e1</serial>
Apr 12 03:31:33 user nova-compute[72081]:     </disk>
Apr 12 03:31:33 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:31:33 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:31:33 user nova-compute[72081]:       <source dev="/dev/sdf"/>
Apr 12 03:31:33 user nova-compute[72081]:       <target dev="vdb" bus="virtio"/>
Apr 12 03:31:33 user nova-compute[72081]:       <serial>0aea0a9e-658d-4acc-ba07-3ef1bace5eb9</serial>
Apr 12 03:31:33 user nova-compute[72081]:     </disk>
Apr 12 03:31:33 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:31:33 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:31:33 user nova-compute[72081]:       <source dev="/dev/sdg"/>
Apr 12 03:31:33 user nova-compute[72081]:       <target dev="vdc" bus="virtio"/>
Apr 12 03:31:33 user nova-compute[72081]:       <serial>6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3</serial>
Apr 12 03:31:33 user nova-compute[72081]:     </disk>
Apr 12 03:31:33 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:31:33 user nova-compute[72081]:       <mac address="fa:16:3e:f1:16:6c"/>
Apr 12 03:31:33 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:31:33 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:31:33 user nova-compute[72081]:       <target dev="tap33b2e7fb-52"/>
Apr 12 03:31:33 user nova-compute[72081]:     </interface>
Apr 12 03:31:33 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:31:33 user nova-compute[72081]:       <mac address="fa:16:3e:10:f4:89"/>
Apr 12 03:31:33 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:31:33 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:31:33 user nova-compute[72081]:       <target dev="tap7740f806-83"/>
Apr 12 03:31:33 user nova-compute[72081]:     </interface>
Apr 12 03:31:33 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:31:33 user nova-compute[72081]:       <mac address="fa:16:3e:15:de:ed"/>
Apr 12 03:31:33 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:31:33 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:31:33 user nova-compute[72081]:       <target dev="tapbe3de447-6d"/>
Apr 12 03:31:33 user nova-compute[72081]:     </interface>
Apr 12 03:31:33 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:31:33 user nova-compute[72081]:       <mac address="fa:16:3e:38:3d:5e"/>
Apr 12 03:31:33 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:31:33 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:31:33 user nova-compute[72081]:       <target dev="tap91177380-5d"/>
Apr 12 03:31:33 user nova-compute[72081]:     </interface>
Apr 12 03:31:33 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:31:33 user nova-compute[72081]:       <mac address="fa:16:3e:02:8f:9e"/>
Apr 12 03:31:33 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:31:33 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:31:33 user nova-compute[72081]:       <target dev="tap41626728-b2"/>
Apr 12 03:31:33 user nova-compute[72081]:     </interface>
Apr 12 03:31:33 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:31:33 user nova-compute[72081]:       <mac address="fa:16:3e:f2:df:e5"/>
Apr 12 03:31:33 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:31:33 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:31:33 user nova-compute[72081]:       <target dev="tap6bb0c89d-0a"/>
Apr 12 03:31:33 user nova-compute[72081]:     </interface>
Apr 12 03:31:33 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:31:33 user nova-compute[72081]:       <mac address="fa:16:3e:db:9a:17"/>
Apr 12 03:31:33 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:31:33 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:31:33 user nova-compute[72081]:       <target dev="tape16cfa52-5c"/>
Apr 12 03:31:33 user nova-compute[72081]:     </interface>
Apr 12 03:31:33 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:31:33 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/43289ac5-3300-4e1f-8d07-2d66ee78e7ba/console.log" append="off"/>
Apr 12 03:31:33 user nova-compute[72081]:     </serial>
Apr 12 03:31:33 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:31:33 user nova-compute[72081]:     <video>
Apr 12 03:31:33 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:31:33 user nova-compute[72081]:     </video>
Apr 12 03:31:33 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:31:33 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:31:33 user nova-compute[72081]:     </rng>
Apr 12 03:31:33 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:31:33 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:31:33 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:31:33 user nova-compute[72081]:     </memballoon>
Apr 12 03:31:33 user nova-compute[72081]:   </devices>
Apr 12 03:31:33 user nova-compute[72081]: </domain>
Apr 12 03:31:33 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:31:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-780207909',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-780207909',id=10,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNCyWVMCVeXTj4eZLmBDk3uEr+ohaWpwIv6zTkIlNZ2QRH2THWN8SXei5drq2IJGQkIvQoNzOXVCuGCaf7h5mScjI1xfzmSUsrhlqHzEQ4MwAG4OF6CGlMBmuH+1KctDlA==',key_name='tempest-keypair-2115967448',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bd808e7e237f4c7490abaff5db0e3529',ramdisk_id='',reservation_id='r-vmalcql5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1194851633',owner_user_name='tempest-TaggedBootDevicesTest-1194851633-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:04Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d29b6ed431d64bcab02c2348feab9aa6',uuid=43289ac5-3300-4e1f-8d07-2d66ee78e7ba,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "33b2e7fb-520f-46d4-be86-e492aff77d4b", "address": "fa:16:3e:f1:16:6c", "network": {"id": "0905eabd-7263-448c-97f6-296c5b42cd24", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-192357488-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap33b2e7fb-52", "ovs_interfaceid": "33b2e7fb-520f-46d4-be86-e492aff77d4b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converting VIF {"id": "33b2e7fb-520f-46d4-be86-e492aff77d4b", "address": "fa:16:3e:f1:16:6c", "network": {"id": "0905eabd-7263-448c-97f6-296c5b42cd24", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-192357488-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap33b2e7fb-52", "ovs_interfaceid": "33b2e7fb-520f-46d4-be86-e492aff77d4b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f1:16:6c,bridge_name='br-int',has_traffic_filtering=True,id=33b2e7fb-520f-46d4-be86-e492aff77d4b,network=Network(0905eabd-7263-448c-97f6-296c5b42cd24),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap33b2e7fb-52') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG os_vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f1:16:6c,bridge_name='br-int',has_traffic_filtering=True,id=33b2e7fb-520f-46d4-be86-e492aff77d4b,network=Network(0905eabd-7263-448c-97f6-296c5b42cd24),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap33b2e7fb-52') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap33b2e7fb-52, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap33b2e7fb-52, col_values=(('external_ids', {'iface-id': '33b2e7fb-520f-46d4-be86-e492aff77d4b', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:f1:16:6c', 'vm-uuid': '43289ac5-3300-4e1f-8d07-2d66ee78e7ba'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: INFO os_vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f1:16:6c,bridge_name='br-int',has_traffic_filtering=True,id=33b2e7fb-520f-46d4-be86-e492aff77d4b,network=Network(0905eabd-7263-448c-97f6-296c5b42cd24),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap33b2e7fb-52')
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:31:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-780207909',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-780207909',id=10,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNCyWVMCVeXTj4eZLmBDk3uEr+ohaWpwIv6zTkIlNZ2QRH2THWN8SXei5drq2IJGQkIvQoNzOXVCuGCaf7h5mScjI1xfzmSUsrhlqHzEQ4MwAG4OF6CGlMBmuH+1KctDlA==',key_name='tempest-keypair-2115967448',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bd808e7e237f4c7490abaff5db0e3529',ramdisk_id='',reservation_id='r-vmalcql5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1194851633',owner_user_name='tempest-TaggedBootDevicesTest-1194851633-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:04Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d29b6ed431d64bcab02c2348feab9aa6',uuid=43289ac5-3300-4e1f-8d07-2d66ee78e7ba,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7740f806-8319-484b-8b0f-bffd8c5fe532", "address": "fa:16:3e:10:f4:89", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.24", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7740f806-83", "ovs_interfaceid": "7740f806-8319-484b-8b0f-bffd8c5fe532", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converting VIF {"id": "7740f806-8319-484b-8b0f-bffd8c5fe532", "address": "fa:16:3e:10:f4:89", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.24", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7740f806-83", "ovs_interfaceid": "7740f806-8319-484b-8b0f-bffd8c5fe532", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:10:f4:89,bridge_name='br-int',has_traffic_filtering=True,id=7740f806-8319-484b-8b0f-bffd8c5fe532,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap7740f806-83') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG os_vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:10:f4:89,bridge_name='br-int',has_traffic_filtering=True,id=7740f806-8319-484b-8b0f-bffd8c5fe532,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap7740f806-83') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap7740f806-83, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap7740f806-83, col_values=(('external_ids', {'iface-id': '7740f806-8319-484b-8b0f-bffd8c5fe532', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:10:f4:89', 'vm-uuid': '43289ac5-3300-4e1f-8d07-2d66ee78e7ba'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: INFO os_vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:10:f4:89,bridge_name='br-int',has_traffic_filtering=True,id=7740f806-8319-484b-8b0f-bffd8c5fe532,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap7740f806-83')
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:31:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-780207909',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-780207909',id=10,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNCyWVMCVeXTj4eZLmBDk3uEr+ohaWpwIv6zTkIlNZ2QRH2THWN8SXei5drq2IJGQkIvQoNzOXVCuGCaf7h5mScjI1xfzmSUsrhlqHzEQ4MwAG4OF6CGlMBmuH+1KctDlA==',key_name='tempest-keypair-2115967448',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bd808e7e237f4c7490abaff5db0e3529',ramdisk_id='',reservation_id='r-vmalcql5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1194851633',owner_user_name='tempest-TaggedBootDevicesTest-1194851633-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:04Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d29b6ed431d64bcab02c2348feab9aa6',uuid=43289ac5-3300-4e1f-8d07-2d66ee78e7ba,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "be3de447-6d22-4fec-b962-0ebe772ed86f", "address": "fa:16:3e:15:de:ed", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.143", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe3de447-6d", "ovs_interfaceid": "be3de447-6d22-4fec-b962-0ebe772ed86f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converting VIF {"id": "be3de447-6d22-4fec-b962-0ebe772ed86f", "address": "fa:16:3e:15:de:ed", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.143", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe3de447-6d", "ovs_interfaceid": "be3de447-6d22-4fec-b962-0ebe772ed86f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:15:de:ed,bridge_name='br-int',has_traffic_filtering=True,id=be3de447-6d22-4fec-b962-0ebe772ed86f,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapbe3de447-6d') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG os_vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:15:de:ed,bridge_name='br-int',has_traffic_filtering=True,id=be3de447-6d22-4fec-b962-0ebe772ed86f,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapbe3de447-6d') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapbe3de447-6d, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapbe3de447-6d, col_values=(('external_ids', {'iface-id': 'be3de447-6d22-4fec-b962-0ebe772ed86f', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:15:de:ed', 'vm-uuid': '43289ac5-3300-4e1f-8d07-2d66ee78e7ba'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: INFO os_vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:15:de:ed,bridge_name='br-int',has_traffic_filtering=True,id=be3de447-6d22-4fec-b962-0ebe772ed86f,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapbe3de447-6d')
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:31:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-780207909',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-780207909',id=10,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNCyWVMCVeXTj4eZLmBDk3uEr+ohaWpwIv6zTkIlNZ2QRH2THWN8SXei5drq2IJGQkIvQoNzOXVCuGCaf7h5mScjI1xfzmSUsrhlqHzEQ4MwAG4OF6CGlMBmuH+1KctDlA==',key_name='tempest-keypair-2115967448',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bd808e7e237f4c7490abaff5db0e3529',ramdisk_id='',reservation_id='r-vmalcql5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1194851633',owner_user_name='tempest-TaggedBootDevicesTest-1194851633-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:04Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d29b6ed431d64bcab02c2348feab9aa6',uuid=43289ac5-3300-4e1f-8d07-2d66ee78e7ba,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "address": "fa:16:3e:38:3d:5e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.54", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap91177380-5d", "ovs_interfaceid": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converting VIF {"id": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "address": "fa:16:3e:38:3d:5e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.54", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap91177380-5d", "ovs_interfaceid": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:38:3d:5e,bridge_name='br-int',has_traffic_filtering=True,id=91177380-5d77-4bed-9ab0-e9085fcb3d5a,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap91177380-5d') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG os_vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:38:3d:5e,bridge_name='br-int',has_traffic_filtering=True,id=91177380-5d77-4bed-9ab0-e9085fcb3d5a,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap91177380-5d') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap91177380-5d, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap91177380-5d, col_values=(('external_ids', {'iface-id': '91177380-5d77-4bed-9ab0-e9085fcb3d5a', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:38:3d:5e', 'vm-uuid': '43289ac5-3300-4e1f-8d07-2d66ee78e7ba'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-plugged-ce9193f1-28a9-4c0f-95ae-e21f8140de19 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-plugged-ce9193f1-28a9-4c0f-95ae-e21f8140de19 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:33 user nova-compute[72081]: WARNING nova.compute.manager [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received unexpected event network-vif-plugged-ce9193f1-28a9-4c0f-95ae-e21f8140de19 for instance with vm_state building and task_state spawning.
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-plugged-835b5a2a-4d35-4d2a-9fdd-0c78b7def163 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-plugged-835b5a2a-4d35-4d2a-9fdd-0c78b7def163 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:33 user nova-compute[72081]: WARNING nova.compute.manager [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received unexpected event network-vif-plugged-835b5a2a-4d35-4d2a-9fdd-0c78b7def163 for instance with vm_state building and task_state spawning.
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-plugged-835b5a2a-4d35-4d2a-9fdd-0c78b7def163 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-plugged-835b5a2a-4d35-4d2a-9fdd-0c78b7def163 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:33 user nova-compute[72081]: WARNING nova.compute.manager [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received unexpected event network-vif-plugged-835b5a2a-4d35-4d2a-9fdd-0c78b7def163 for instance with vm_state building and task_state spawning.
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-plugged-4e57de22-e876-4bfe-bd07-a1003afd2295 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-plugged-4e57de22-e876-4bfe-bd07-a1003afd2295 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:33 user nova-compute[72081]: WARNING nova.compute.manager [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received unexpected event network-vif-plugged-4e57de22-e876-4bfe-bd07-a1003afd2295 for instance with vm_state building and task_state spawning.
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-plugged-4e57de22-e876-4bfe-bd07-a1003afd2295 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-plugged-4e57de22-e876-4bfe-bd07-a1003afd2295 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:33 user nova-compute[72081]: WARNING nova.compute.manager [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received unexpected event network-vif-plugged-4e57de22-e876-4bfe-bd07-a1003afd2295 for instance with vm_state building and task_state spawning.
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-plugged-21860860-2a08-43dc-bda8-0a1d0fab0634 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-plugged-21860860-2a08-43dc-bda8-0a1d0fab0634 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:33 user nova-compute[72081]: WARNING nova.compute.manager [req-f73e806d-e826-406a-9788-fa6fcb257bbb req-2d840686-2cfa-4ecb-8e60-0832bfcb9a1c service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received unexpected event network-vif-plugged-21860860-2a08-43dc-bda8-0a1d0fab0634 for instance with vm_state building and task_state spawning.
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:34 user nova-compute[72081]: INFO os_vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:38:3d:5e,bridge_name='br-int',has_traffic_filtering=True,id=91177380-5d77-4bed-9ab0-e9085fcb3d5a,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap91177380-5d')
Apr 12 03:31:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:31:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-780207909',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-780207909',id=10,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNCyWVMCVeXTj4eZLmBDk3uEr+ohaWpwIv6zTkIlNZ2QRH2THWN8SXei5drq2IJGQkIvQoNzOXVCuGCaf7h5mScjI1xfzmSUsrhlqHzEQ4MwAG4OF6CGlMBmuH+1KctDlA==',key_name='tempest-keypair-2115967448',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bd808e7e237f4c7490abaff5db0e3529',ramdisk_id='',reservation_id='r-vmalcql5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1194851633',owner_user_name='tempest-TaggedBootDevicesTest-1194851633-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:04Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d29b6ed431d64bcab02c2348feab9aa6',uuid=43289ac5-3300-4e1f-8d07-2d66ee78e7ba,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "address": "fa:16:3e:02:8f:9e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.17", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap41626728-b2", "ovs_interfaceid": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converting VIF {"id": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "address": "fa:16:3e:02:8f:9e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.17", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap41626728-b2", "ovs_interfaceid": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:02:8f:9e,bridge_name='br-int',has_traffic_filtering=True,id=41626728-b21d-4ef8-9d86-5328d2feb3f9,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap41626728-b2') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG os_vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:02:8f:9e,bridge_name='br-int',has_traffic_filtering=True,id=41626728-b21d-4ef8-9d86-5328d2feb3f9,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap41626728-b2') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap41626728-b2, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap41626728-b2, col_values=(('external_ids', {'iface-id': '41626728-b21d-4ef8-9d86-5328d2feb3f9', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:02:8f:9e', 'vm-uuid': '43289ac5-3300-4e1f-8d07-2d66ee78e7ba'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:34 user nova-compute[72081]: INFO os_vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:02:8f:9e,bridge_name='br-int',has_traffic_filtering=True,id=41626728-b21d-4ef8-9d86-5328d2feb3f9,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap41626728-b2')
Apr 12 03:31:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:31:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-780207909',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-780207909',id=10,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNCyWVMCVeXTj4eZLmBDk3uEr+ohaWpwIv6zTkIlNZ2QRH2THWN8SXei5drq2IJGQkIvQoNzOXVCuGCaf7h5mScjI1xfzmSUsrhlqHzEQ4MwAG4OF6CGlMBmuH+1KctDlA==',key_name='tempest-keypair-2115967448',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bd808e7e237f4c7490abaff5db0e3529',ramdisk_id='',reservation_id='r-vmalcql5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1194851633',owner_user_name='tempest-TaggedBootDevicesTest-1194851633-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:04Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d29b6ed431d64bcab02c2348feab9aa6',uuid=43289ac5-3300-4e1f-8d07-2d66ee78e7ba,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "address": "fa:16:3e:f2:df:e5", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bb0c89d-0a", "ovs_interfaceid": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converting VIF {"id": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "address": "fa:16:3e:f2:df:e5", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bb0c89d-0a", "ovs_interfaceid": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f2:df:e5,bridge_name='br-int',has_traffic_filtering=True,id=6bb0c89d-0a60-4413-b09d-08d4d6ea3a52,network=Network(2767acc3-5e96-41ad-a7f4-fbd5b4acce3a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6bb0c89d-0a') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG os_vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f2:df:e5,bridge_name='br-int',has_traffic_filtering=True,id=6bb0c89d-0a60-4413-b09d-08d4d6ea3a52,network=Network(2767acc3-5e96-41ad-a7f4-fbd5b4acce3a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6bb0c89d-0a') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap6bb0c89d-0a, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap6bb0c89d-0a, col_values=(('external_ids', {'iface-id': '6bb0c89d-0a60-4413-b09d-08d4d6ea3a52', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:f2:df:e5', 'vm-uuid': '43289ac5-3300-4e1f-8d07-2d66ee78e7ba'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:34 user nova-compute[72081]: INFO os_vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f2:df:e5,bridge_name='br-int',has_traffic_filtering=True,id=6bb0c89d-0a60-4413-b09d-08d4d6ea3a52,network=Network(2767acc3-5e96-41ad-a7f4-fbd5b4acce3a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6bb0c89d-0a')
Apr 12 03:31:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:31:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-780207909',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-780207909',id=10,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNCyWVMCVeXTj4eZLmBDk3uEr+ohaWpwIv6zTkIlNZ2QRH2THWN8SXei5drq2IJGQkIvQoNzOXVCuGCaf7h5mScjI1xfzmSUsrhlqHzEQ4MwAG4OF6CGlMBmuH+1KctDlA==',key_name='tempest-keypair-2115967448',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bd808e7e237f4c7490abaff5db0e3529',ramdisk_id='',reservation_id='r-vmalcql5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1194851633',owner_user_name='tempest-TaggedBootDevicesTest-1194851633-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:31:04Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d29b6ed431d64bcab02c2348feab9aa6',uuid=43289ac5-3300-4e1f-8d07-2d66ee78e7ba,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "address": "fa:16:3e:db:9a:17", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16cfa52-5c", "ovs_interfaceid": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converting VIF {"id": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "address": "fa:16:3e:db:9a:17", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16cfa52-5c", "ovs_interfaceid": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:db:9a:17,bridge_name='br-int',has_traffic_filtering=True,id=e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1,network=Network(2767acc3-5e96-41ad-a7f4-fbd5b4acce3a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16cfa52-5c') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG os_vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:db:9a:17,bridge_name='br-int',has_traffic_filtering=True,id=e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1,network=Network(2767acc3-5e96-41ad-a7f4-fbd5b4acce3a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16cfa52-5c') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tape16cfa52-5c, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tape16cfa52-5c, col_values=(('external_ids', {'iface-id': 'e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:db:9a:17', 'vm-uuid': '43289ac5-3300-4e1f-8d07-2d66ee78e7ba'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:34 user nova-compute[72081]: INFO os_vif [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:db:9a:17,bridge_name='br-int',has_traffic_filtering=True,id=e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1,network=Network(2767acc3-5e96-41ad-a7f4-fbd5b4acce3a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16cfa52-5c')
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] No BDM found with device name hda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] No BDM found with device name vdc, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] No VIF found with MAC fa:16:3e:f1:16:6c, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] No VIF found with MAC fa:16:3e:02:8f:9e, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:31:34 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Using config drive
Apr 12 03:31:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:34 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Creating config drive at /opt/stack/data/nova/instances/43289ac5-3300-4e1f-8d07-2d66ee78e7ba/disk.config
Apr 12 03:31:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Running cmd (subprocess): genisoimage -o /opt/stack/data/nova/instances/43289ac5-3300-4e1f-8d07-2d66ee78e7ba/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmptbpclhkh {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] CMD "genisoimage -o /opt/stack/data/nova/instances/43289ac5-3300-4e1f-8d07-2d66ee78e7ba/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmptbpclhkh" returned: 0 in 0.051s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG nova.compute.manager [req-bb2c5c71-c77b-4bbb-89ef-73756e266f79 req-eb490d8a-95ca-447e-92ec-83b94e387f0e service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-plugged-ac401bfa-a662-4290-af6e-0a39e8fde61c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bb2c5c71-c77b-4bbb-89ef-73756e266f79 req-eb490d8a-95ca-447e-92ec-83b94e387f0e service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bb2c5c71-c77b-4bbb-89ef-73756e266f79 req-eb490d8a-95ca-447e-92ec-83b94e387f0e service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bb2c5c71-c77b-4bbb-89ef-73756e266f79 req-eb490d8a-95ca-447e-92ec-83b94e387f0e service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG nova.compute.manager [req-bb2c5c71-c77b-4bbb-89ef-73756e266f79 req-eb490d8a-95ca-447e-92ec-83b94e387f0e service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-plugged-ac401bfa-a662-4290-af6e-0a39e8fde61c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:34 user nova-compute[72081]: WARNING nova.compute.manager [req-bb2c5c71-c77b-4bbb-89ef-73756e266f79 req-eb490d8a-95ca-447e-92ec-83b94e387f0e service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received unexpected event network-vif-plugged-ac401bfa-a662-4290-af6e-0a39e8fde61c for instance with vm_state building and task_state spawning.
Apr 12 03:31:34 user nova-compute[72081]: DEBUG nova.compute.manager [req-bb2c5c71-c77b-4bbb-89ef-73756e266f79 req-eb490d8a-95ca-447e-92ec-83b94e387f0e service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-plugged-8248913f-46c8-4f3d-ab47-0203cb267169 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bb2c5c71-c77b-4bbb-89ef-73756e266f79 req-eb490d8a-95ca-447e-92ec-83b94e387f0e service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bb2c5c71-c77b-4bbb-89ef-73756e266f79 req-eb490d8a-95ca-447e-92ec-83b94e387f0e service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bb2c5c71-c77b-4bbb-89ef-73756e266f79 req-eb490d8a-95ca-447e-92ec-83b94e387f0e service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG nova.compute.manager [req-bb2c5c71-c77b-4bbb-89ef-73756e266f79 req-eb490d8a-95ca-447e-92ec-83b94e387f0e service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-plugged-8248913f-46c8-4f3d-ab47-0203cb267169 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:34 user nova-compute[72081]: WARNING nova.compute.manager [req-bb2c5c71-c77b-4bbb-89ef-73756e266f79 req-eb490d8a-95ca-447e-92ec-83b94e387f0e service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received unexpected event network-vif-plugged-8248913f-46c8-4f3d-ab47-0203cb267169 for instance with vm_state building and task_state spawning.
Apr 12 03:31:34 user nova-compute[72081]: DEBUG nova.compute.manager [req-bb2c5c71-c77b-4bbb-89ef-73756e266f79 req-eb490d8a-95ca-447e-92ec-83b94e387f0e service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-plugged-8248913f-46c8-4f3d-ab47-0203cb267169 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bb2c5c71-c77b-4bbb-89ef-73756e266f79 req-eb490d8a-95ca-447e-92ec-83b94e387f0e service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bb2c5c71-c77b-4bbb-89ef-73756e266f79 req-eb490d8a-95ca-447e-92ec-83b94e387f0e service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bb2c5c71-c77b-4bbb-89ef-73756e266f79 req-eb490d8a-95ca-447e-92ec-83b94e387f0e service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG nova.compute.manager [req-bb2c5c71-c77b-4bbb-89ef-73756e266f79 req-eb490d8a-95ca-447e-92ec-83b94e387f0e service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-plugged-8248913f-46c8-4f3d-ab47-0203cb267169 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:34 user nova-compute[72081]: WARNING nova.compute.manager [req-bb2c5c71-c77b-4bbb-89ef-73756e266f79 req-eb490d8a-95ca-447e-92ec-83b94e387f0e service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received unexpected event network-vif-plugged-8248913f-46c8-4f3d-ab47-0203cb267169 for instance with vm_state building and task_state spawning.
Apr 12 03:31:34 user nova-compute[72081]: DEBUG nova.compute.manager [req-8f2c60b2-8885-44e5-a63f-08b2dda2452a req-5642d154-64e1-4870-a85c-9627bf9eef5f service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-plugged-a1aaff41-e246-4887-89c6-a91163d4efd5 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8f2c60b2-8885-44e5-a63f-08b2dda2452a req-5642d154-64e1-4870-a85c-9627bf9eef5f service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8f2c60b2-8885-44e5-a63f-08b2dda2452a req-5642d154-64e1-4870-a85c-9627bf9eef5f service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8f2c60b2-8885-44e5-a63f-08b2dda2452a req-5642d154-64e1-4870-a85c-9627bf9eef5f service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:34 user nova-compute[72081]: DEBUG nova.compute.manager [req-8f2c60b2-8885-44e5-a63f-08b2dda2452a req-5642d154-64e1-4870-a85c-9627bf9eef5f service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-plugged-a1aaff41-e246-4887-89c6-a91163d4efd5 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:34 user nova-compute[72081]: WARNING nova.compute.manager [req-8f2c60b2-8885-44e5-a63f-08b2dda2452a req-5642d154-64e1-4870-a85c-9627bf9eef5f service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received unexpected event network-vif-plugged-a1aaff41-e246-4887-89c6-a91163d4efd5 for instance with vm_state building and task_state spawning.
Apr 12 03:31:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:35 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270295.946503, 7119b8b3-291b-4868-820e-d7bd3dd3ada4 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:31:35 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] VM Resumed (Lifecycle Event)
Apr 12 03:31:35 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:31:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:31:35 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:31:35 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Instance spawned successfully.
Apr 12 03:31:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:31:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:36 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:31:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:31:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:31:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:31:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:31:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:31:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:31:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:36 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:31:36 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270295.9468808, 7119b8b3-291b-4868-820e-d7bd3dd3ada4 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:31:36 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] VM Started (Lifecycle Event)
Apr 12 03:31:36 user nova-compute[72081]: DEBUG nova.compute.manager [req-dfd8e0b3-da22-48a2-aac6-8bea802411a3 req-9699ee1f-c485-46fa-8c21-b9f661f44496 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-plugged-21860860-2a08-43dc-bda8-0a1d0fab0634 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-dfd8e0b3-da22-48a2-aac6-8bea802411a3 req-9699ee1f-c485-46fa-8c21-b9f661f44496 service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-dfd8e0b3-da22-48a2-aac6-8bea802411a3 req-9699ee1f-c485-46fa-8c21-b9f661f44496 service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-dfd8e0b3-da22-48a2-aac6-8bea802411a3 req-9699ee1f-c485-46fa-8c21-b9f661f44496 service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:36 user nova-compute[72081]: DEBUG nova.compute.manager [req-dfd8e0b3-da22-48a2-aac6-8bea802411a3 req-9699ee1f-c485-46fa-8c21-b9f661f44496 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-plugged-21860860-2a08-43dc-bda8-0a1d0fab0634 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:36 user nova-compute[72081]: WARNING nova.compute.manager [req-dfd8e0b3-da22-48a2-aac6-8bea802411a3 req-9699ee1f-c485-46fa-8c21-b9f661f44496 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received unexpected event network-vif-plugged-21860860-2a08-43dc-bda8-0a1d0fab0634 for instance with vm_state building and task_state spawning.
Apr 12 03:31:36 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:31:36 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:31:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:36 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:31:36 user nova-compute[72081]: INFO nova.compute.manager [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Took 19.00 seconds to spawn the instance on the hypervisor.
Apr 12 03:31:36 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:31:36 user nova-compute[72081]: INFO nova.compute.manager [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Took 38.17 seconds to build instance.
Apr 12 03:31:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b0f2113c-5471-4e87-8770-726a18efa525 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 38.273s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-3d1033b6-012d-4f84-b058-943d987177da req-91a9bc01-5d6c-4116-8306-f2376f1e2f76 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-plugged-be3de447-6d22-4fec-b962-0ebe772ed86f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3d1033b6-012d-4f84-b058-943d987177da req-91a9bc01-5d6c-4116-8306-f2376f1e2f76 service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3d1033b6-012d-4f84-b058-943d987177da req-91a9bc01-5d6c-4116-8306-f2376f1e2f76 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3d1033b6-012d-4f84-b058-943d987177da req-91a9bc01-5d6c-4116-8306-f2376f1e2f76 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-3d1033b6-012d-4f84-b058-943d987177da req-91a9bc01-5d6c-4116-8306-f2376f1e2f76 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-plugged-be3de447-6d22-4fec-b962-0ebe772ed86f {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:37 user nova-compute[72081]: WARNING nova.compute.manager [req-3d1033b6-012d-4f84-b058-943d987177da req-91a9bc01-5d6c-4116-8306-f2376f1e2f76 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received unexpected event network-vif-plugged-be3de447-6d22-4fec-b962-0ebe772ed86f for instance with vm_state building and task_state spawning.
Apr 12 03:31:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-8f401b5d-2169-41d9-8818-885ab51ab63d req-0783f96e-e4f3-48a0-92ce-8ff78c51f611 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-plugged-41626728-b21d-4ef8-9d86-5328d2feb3f9 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8f401b5d-2169-41d9-8818-885ab51ab63d req-0783f96e-e4f3-48a0-92ce-8ff78c51f611 service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8f401b5d-2169-41d9-8818-885ab51ab63d req-0783f96e-e4f3-48a0-92ce-8ff78c51f611 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8f401b5d-2169-41d9-8818-885ab51ab63d req-0783f96e-e4f3-48a0-92ce-8ff78c51f611 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-8f401b5d-2169-41d9-8818-885ab51ab63d req-0783f96e-e4f3-48a0-92ce-8ff78c51f611 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-plugged-41626728-b21d-4ef8-9d86-5328d2feb3f9 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:37 user nova-compute[72081]: WARNING nova.compute.manager [req-8f401b5d-2169-41d9-8818-885ab51ab63d req-0783f96e-e4f3-48a0-92ce-8ff78c51f611 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received unexpected event network-vif-plugged-41626728-b21d-4ef8-9d86-5328d2feb3f9 for instance with vm_state building and task_state spawning.
Apr 12 03:31:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-plugged-33b2e7fb-520f-46d4-be86-e492aff77d4b {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-plugged-33b2e7fb-520f-46d4-be86-e492aff77d4b {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:38 user nova-compute[72081]: WARNING nova.compute.manager [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received unexpected event network-vif-plugged-33b2e7fb-520f-46d4-be86-e492aff77d4b for instance with vm_state building and task_state spawning.
Apr 12 03:31:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-plugged-33b2e7fb-520f-46d4-be86-e492aff77d4b {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-plugged-33b2e7fb-520f-46d4-be86-e492aff77d4b {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:38 user nova-compute[72081]: WARNING nova.compute.manager [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received unexpected event network-vif-plugged-33b2e7fb-520f-46d4-be86-e492aff77d4b for instance with vm_state building and task_state spawning.
Apr 12 03:31:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-plugged-7740f806-8319-484b-8b0f-bffd8c5fe532 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-plugged-7740f806-8319-484b-8b0f-bffd8c5fe532 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:38 user nova-compute[72081]: WARNING nova.compute.manager [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received unexpected event network-vif-plugged-7740f806-8319-484b-8b0f-bffd8c5fe532 for instance with vm_state building and task_state spawning.
Apr 12 03:31:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-plugged-7740f806-8319-484b-8b0f-bffd8c5fe532 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-plugged-7740f806-8319-484b-8b0f-bffd8c5fe532 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:38 user nova-compute[72081]: WARNING nova.compute.manager [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received unexpected event network-vif-plugged-7740f806-8319-484b-8b0f-bffd8c5fe532 for instance with vm_state building and task_state spawning.
Apr 12 03:31:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-plugged-91177380-5d77-4bed-9ab0-e9085fcb3d5a {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-plugged-91177380-5d77-4bed-9ab0-e9085fcb3d5a {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:38 user nova-compute[72081]: WARNING nova.compute.manager [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received unexpected event network-vif-plugged-91177380-5d77-4bed-9ab0-e9085fcb3d5a for instance with vm_state building and task_state spawning.
Apr 12 03:31:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-plugged-91177380-5d77-4bed-9ab0-e9085fcb3d5a {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-plugged-91177380-5d77-4bed-9ab0-e9085fcb3d5a {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:38 user nova-compute[72081]: WARNING nova.compute.manager [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received unexpected event network-vif-plugged-91177380-5d77-4bed-9ab0-e9085fcb3d5a for instance with vm_state building and task_state spawning.
Apr 12 03:31:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-plugged-6bb0c89d-0a60-4413-b09d-08d4d6ea3a52 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-plugged-6bb0c89d-0a60-4413-b09d-08d4d6ea3a52 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:38 user nova-compute[72081]: WARNING nova.compute.manager [req-39b147e8-c780-4481-985e-25dd30ae2d96 req-ab277d77-7372-4dbe-8ab9-ce7f1d3c7f76 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received unexpected event network-vif-plugged-6bb0c89d-0a60-4413-b09d-08d4d6ea3a52 for instance with vm_state building and task_state spawning.
Apr 12 03:31:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:39 user nova-compute[72081]: DEBUG nova.compute.manager [req-d449df27-14bd-44bc-ac66-cf7e2e93d327 req-99792d73-24c6-49b6-92b5-79b62d339113 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-plugged-be3de447-6d22-4fec-b962-0ebe772ed86f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d449df27-14bd-44bc-ac66-cf7e2e93d327 req-99792d73-24c6-49b6-92b5-79b62d339113 service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d449df27-14bd-44bc-ac66-cf7e2e93d327 req-99792d73-24c6-49b6-92b5-79b62d339113 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d449df27-14bd-44bc-ac66-cf7e2e93d327 req-99792d73-24c6-49b6-92b5-79b62d339113 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:39 user nova-compute[72081]: DEBUG nova.compute.manager [req-d449df27-14bd-44bc-ac66-cf7e2e93d327 req-99792d73-24c6-49b6-92b5-79b62d339113 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-plugged-be3de447-6d22-4fec-b962-0ebe772ed86f {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:39 user nova-compute[72081]: WARNING nova.compute.manager [req-d449df27-14bd-44bc-ac66-cf7e2e93d327 req-99792d73-24c6-49b6-92b5-79b62d339113 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received unexpected event network-vif-plugged-be3de447-6d22-4fec-b962-0ebe772ed86f for instance with vm_state building and task_state spawning.
Apr 12 03:31:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:39 user nova-compute[72081]: DEBUG nova.compute.manager [req-7c9b10fa-4758-473d-8983-be154863b243 req-120fd041-9e1c-4333-92d9-1b6070006e74 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-plugged-41626728-b21d-4ef8-9d86-5328d2feb3f9 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7c9b10fa-4758-473d-8983-be154863b243 req-120fd041-9e1c-4333-92d9-1b6070006e74 service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7c9b10fa-4758-473d-8983-be154863b243 req-120fd041-9e1c-4333-92d9-1b6070006e74 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7c9b10fa-4758-473d-8983-be154863b243 req-120fd041-9e1c-4333-92d9-1b6070006e74 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:39 user nova-compute[72081]: DEBUG nova.compute.manager [req-7c9b10fa-4758-473d-8983-be154863b243 req-120fd041-9e1c-4333-92d9-1b6070006e74 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-plugged-41626728-b21d-4ef8-9d86-5328d2feb3f9 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:39 user nova-compute[72081]: WARNING nova.compute.manager [req-7c9b10fa-4758-473d-8983-be154863b243 req-120fd041-9e1c-4333-92d9-1b6070006e74 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received unexpected event network-vif-plugged-41626728-b21d-4ef8-9d86-5328d2feb3f9 for instance with vm_state building and task_state spawning.
Apr 12 03:31:40 user nova-compute[72081]: DEBUG nova.compute.manager [req-12e64648-54c3-4c2f-a842-4eff1d20c5de req-2e8c793b-f83f-483d-9a88-da18225ae76c service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-plugged-6bb0c89d-0a60-4413-b09d-08d4d6ea3a52 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-12e64648-54c3-4c2f-a842-4eff1d20c5de req-2e8c793b-f83f-483d-9a88-da18225ae76c service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-12e64648-54c3-4c2f-a842-4eff1d20c5de req-2e8c793b-f83f-483d-9a88-da18225ae76c service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-12e64648-54c3-4c2f-a842-4eff1d20c5de req-2e8c793b-f83f-483d-9a88-da18225ae76c service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:40 user nova-compute[72081]: DEBUG nova.compute.manager [req-12e64648-54c3-4c2f-a842-4eff1d20c5de req-2e8c793b-f83f-483d-9a88-da18225ae76c service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-plugged-6bb0c89d-0a60-4413-b09d-08d4d6ea3a52 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:40 user nova-compute[72081]: WARNING nova.compute.manager [req-12e64648-54c3-4c2f-a842-4eff1d20c5de req-2e8c793b-f83f-483d-9a88-da18225ae76c service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received unexpected event network-vif-plugged-6bb0c89d-0a60-4413-b09d-08d4d6ea3a52 for instance with vm_state building and task_state spawning.
Apr 12 03:31:40 user nova-compute[72081]: DEBUG nova.compute.manager [req-12e64648-54c3-4c2f-a842-4eff1d20c5de req-2e8c793b-f83f-483d-9a88-da18225ae76c service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-plugged-e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-12e64648-54c3-4c2f-a842-4eff1d20c5de req-2e8c793b-f83f-483d-9a88-da18225ae76c service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-12e64648-54c3-4c2f-a842-4eff1d20c5de req-2e8c793b-f83f-483d-9a88-da18225ae76c service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-12e64648-54c3-4c2f-a842-4eff1d20c5de req-2e8c793b-f83f-483d-9a88-da18225ae76c service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:40 user nova-compute[72081]: DEBUG nova.compute.manager [req-12e64648-54c3-4c2f-a842-4eff1d20c5de req-2e8c793b-f83f-483d-9a88-da18225ae76c service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-plugged-e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:40 user nova-compute[72081]: WARNING nova.compute.manager [req-12e64648-54c3-4c2f-a842-4eff1d20c5de req-2e8c793b-f83f-483d-9a88-da18225ae76c service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received unexpected event network-vif-plugged-e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1 for instance with vm_state building and task_state spawning.
Apr 12 03:31:40 user nova-compute[72081]: DEBUG nova.compute.manager [req-12e64648-54c3-4c2f-a842-4eff1d20c5de req-2e8c793b-f83f-483d-9a88-da18225ae76c service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-plugged-e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:31:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-12e64648-54c3-4c2f-a842-4eff1d20c5de req-2e8c793b-f83f-483d-9a88-da18225ae76c service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-12e64648-54c3-4c2f-a842-4eff1d20c5de req-2e8c793b-f83f-483d-9a88-da18225ae76c service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-12e64648-54c3-4c2f-a842-4eff1d20c5de req-2e8c793b-f83f-483d-9a88-da18225ae76c service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:40 user nova-compute[72081]: DEBUG nova.compute.manager [req-12e64648-54c3-4c2f-a842-4eff1d20c5de req-2e8c793b-f83f-483d-9a88-da18225ae76c service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-plugged-e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:31:40 user nova-compute[72081]: WARNING nova.compute.manager [req-12e64648-54c3-4c2f-a842-4eff1d20c5de req-2e8c793b-f83f-483d-9a88-da18225ae76c service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received unexpected event network-vif-plugged-e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1 for instance with vm_state building and task_state spawning.
Apr 12 03:31:40 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270300.3392417, 43289ac5-3300-4e1f-8d07-2d66ee78e7ba => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:31:40 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] VM Resumed (Lifecycle Event)
Apr 12 03:31:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:31:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:31:40 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Instance spawned successfully.
Apr 12 03:31:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:31:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:31:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:31:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:31:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:31:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:31:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:31:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:31:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:31:40 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:31:40 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270300.3416433, 43289ac5-3300-4e1f-8d07-2d66ee78e7ba => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:31:40 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] VM Started (Lifecycle Event)
Apr 12 03:31:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:31:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:31:40 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:31:40 user nova-compute[72081]: INFO nova.compute.manager [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Took 21.82 seconds to spawn the instance on the hypervisor.
Apr 12 03:31:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:31:40 user nova-compute[72081]: INFO nova.compute.manager [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Took 37.01 seconds to build instance.
Apr 12 03:31:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0e91d655-ea02-48e5-826a-bd7f4a532d54 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 37.110s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:54 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:31:55 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:31:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:57 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:31:57 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:31:57 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:31:58 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:31:58 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:31:58 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:31:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:31:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:31:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:31:58 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:31:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk --force-share --output=json" returned: 0 in 0.129s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:31:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdf (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:31:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdg (vdc) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:31:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:31:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7/disk --force-share --output=json" returned: 0 in 0.150s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:31:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:31:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ef4ab4cb-6164-47ea-be1d-cbe0698f991d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ef4ab4cb-6164-47ea-be1d-cbe0698f991d/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ef4ab4cb-6164-47ea-be1d-cbe0698f991d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Acquiring lock "f728398d-524b-4f22-b91b-a431d62a952c" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "f728398d-524b-4f22-b91b-a431d62a952c" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:00 user nova-compute[72081]: DEBUG nova.compute.manager [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:32:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ef4ab4cb-6164-47ea-be1d-cbe0698f991d/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f1a0f3cf-1004-4ff9-9e41-d03291abbb9a/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.012s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:00 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:32:00 user nova-compute[72081]: INFO nova.compute.claims [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Claim successful on node user
Apr 12 03:32:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f1a0f3cf-1004-4ff9-9e41-d03291abbb9a/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f1a0f3cf-1004-4ff9-9e41-d03291abbb9a/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f1a0f3cf-1004-4ff9-9e41-d03291abbb9a/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0/disk --force-share --output=json" returned: 0 in 0.160s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:01 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:32:01 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:32:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.602s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:01 user nova-compute[72081]: DEBUG nova.compute.manager [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:32:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:01 user nova-compute[72081]: DEBUG nova.compute.manager [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:32:01 user nova-compute[72081]: DEBUG nova.network.neutron [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:32:01 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:32:01 user nova-compute[72081]: DEBUG nova.compute.manager [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:32:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:01 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:32:01 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:32:01 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdd (vdc) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:32:01 user nova-compute[72081]: DEBUG nova.policy [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '3da7394c51664a7caacfda6b7dda9066', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '83a7ef3f97fd47849973d4e4482fb5f9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.compute.manager [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:32:02 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Creating image(s)
Apr 12 03:32:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Acquiring lock "/opt/stack/data/nova/instances/f728398d-524b-4f22-b91b-a431d62a952c/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "/opt/stack/data/nova/instances/f728398d-524b-4f22-b91b-a431d62a952c/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "/opt/stack/data/nova/instances/f728398d-524b-4f22-b91b-a431d62a952c/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.network.neutron [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Successfully created port: b0727a17-1b28-4bd2-a4bd-ba6f779fb19b {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:32:02 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:32:02 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=7329MB free_disk=26.394298553466797GB free_vcpus=1 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.compute.manager [req-007b8e88-0035-4732-8f58-af3d83d08c4c req-e29ce0a9-7d0c-4ddf-9514-a943c9f9552c service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Received event network-changed-540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.compute.manager [req-007b8e88-0035-4732-8f58-af3d83d08c4c req-e29ce0a9-7d0c-4ddf-9514-a943c9f9552c service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Refreshing instance network info cache due to event network-changed-540b7307-8fea-4b50-bab8-626983135a96. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-007b8e88-0035-4732-8f58-af3d83d08c4c req-e29ce0a9-7d0c-4ddf-9514-a943c9f9552c service nova] Acquiring lock "refresh_cache-b23d4bcd-989a-45e1-89a3-6e3d30229a32" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-007b8e88-0035-4732-8f58-af3d83d08c4c req-e29ce0a9-7d0c-4ddf-9514-a943c9f9552c service nova] Acquired lock "refresh_cache-b23d4bcd-989a-45e1-89a3-6e3d30229a32" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.network.neutron [req-007b8e88-0035-4732-8f58-af3d83d08c4c req-e29ce0a9-7d0c-4ddf-9514-a943c9f9552c service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Refreshing network info cache for port 540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.236s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance ef4ab4cb-6164-47ea-be1d-cbe0698f991d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 2d0904c3-74e0-47bb-a832-758a7035b2f5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 7119b8b3-291b-4868-820e-d7bd3dd3ada4 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance b23d4bcd-989a-45e1-89a3-6e3d30229a32 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 56834002-7593-4592-a86d-23bb90d00be0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance f728398d-524b-4f22-b91b-a431d62a952c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 413850ca-8175-4ed6-a6cf-ff42230fa1e0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 72ba1218-065d-40c4-8503-7c4155e47503 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 43289ac5-3300-4e1f-8d07-2d66ee78e7ba actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance ccf9f424-d230-4079-87c5-bb08bc85b0b7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance f1a0f3cf-1004-4ff9-9e41-d03291abbb9a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 12 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=2048MB phys_disk=40GB used_disk=10GB total_vcpus=12 used_vcpus=12 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.compute.manager [None req-72eaa388-fa0b-4ce8-8695-6b1901c05616 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.153s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/f728398d-524b-4f22-b91b-a431d62a952c/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:02 user nova-compute[72081]: INFO nova.compute.manager [None req-72eaa388-fa0b-4ce8-8695-6b1901c05616 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] instance snapshotting
Apr 12 03:32:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/f728398d-524b-4f22-b91b-a431d62a952c/disk 1073741824" returned: 0 in 0.063s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.219s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:02 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-72eaa388-fa0b-4ce8-8695-6b1901c05616 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Beginning live snapshot process
Apr 12 03:32:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.150s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Checking if we can resize image /opt/stack/data/nova/instances/f728398d-524b-4f22-b91b-a431d62a952c/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f728398d-524b-4f22-b91b-a431d62a952c/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f728398d-524b-4f22-b91b-a431d62a952c/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Cannot resize image /opt/stack/data/nova/instances/f728398d-524b-4f22-b91b-a431d62a952c/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.objects.instance [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lazy-loading 'migration_context' on Instance uuid f728398d-524b-4f22-b91b-a431d62a952c {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Ensure instance console log exists: /opt/stack/data/nova/instances/f728398d-524b-4f22-b91b-a431d62a952c/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.759s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-72eaa388-fa0b-4ce8-8695-6b1901c05616 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json -f qcow2 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG nova.network.neutron [req-007b8e88-0035-4732-8f58-af3d83d08c4c req-e29ce0a9-7d0c-4ddf-9514-a943c9f9552c service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Updated VIF entry in instance network info cache for port 540b7307-8fea-4b50-bab8-626983135a96. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG nova.network.neutron [req-007b8e88-0035-4732-8f58-af3d83d08c4c req-e29ce0a9-7d0c-4ddf-9514-a943c9f9552c service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Updating instance_info_cache with network_info: [{"id": "540b7307-8fea-4b50-bab8-626983135a96", "address": "fa:16:3e:8d:a9:ab", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.135", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap540b7307-8f", "ovs_interfaceid": "540b7307-8fea-4b50-bab8-626983135a96", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-72eaa388-fa0b-4ce8-8695-6b1901c05616 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json -f qcow2" returned: 0 in 0.160s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-72eaa388-fa0b-4ce8-8695-6b1901c05616 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json -f qcow2 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-007b8e88-0035-4732-8f58-af3d83d08c4c req-e29ce0a9-7d0c-4ddf-9514-a943c9f9552c service nova] Releasing lock "refresh_cache-b23d4bcd-989a-45e1-89a3-6e3d30229a32" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-72eaa388-fa0b-4ce8-8695-6b1901c05616 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json -f qcow2" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-72eaa388-fa0b-4ce8-8695-6b1901c05616 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG nova.network.neutron [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Successfully updated port: b0727a17-1b28-4bd2-a4bd-ba6f779fb19b {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-4d0fd191-b47f-42e6-bc6e-5a63ca689498 req-287284fb-bc04-4c07-886e-7663796fd558 service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Received event network-changed-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-4d0fd191-b47f-42e6-bc6e-5a63ca689498 req-287284fb-bc04-4c07-886e-7663796fd558 service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Refreshing instance network info cache due to event network-changed-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4d0fd191-b47f-42e6-bc6e-5a63ca689498 req-287284fb-bc04-4c07-886e-7663796fd558 service nova] Acquiring lock "refresh_cache-f728398d-524b-4f22-b91b-a431d62a952c" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4d0fd191-b47f-42e6-bc6e-5a63ca689498 req-287284fb-bc04-4c07-886e-7663796fd558 service nova] Acquired lock "refresh_cache-f728398d-524b-4f22-b91b-a431d62a952c" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG nova.network.neutron [req-4d0fd191-b47f-42e6-bc6e-5a63ca689498 req-287284fb-bc04-4c07-886e-7663796fd558 service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Refreshing network info cache for port b0727a17-1b28-4bd2-a4bd-ba6f779fb19b {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Acquiring lock "refresh_cache-f728398d-524b-4f22-b91b-a431d62a952c" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG nova.network.neutron [req-4d0fd191-b47f-42e6-bc6e-5a63ca689498 req-287284fb-bc04-4c07-886e-7663796fd558 service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-72eaa388-fa0b-4ce8-8695-6b1901c05616 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-72eaa388-fa0b-4ce8-8695-6b1901c05616 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmppqdjwf37/3d6edbcf86824f0da598a5191c6c3caa.delta 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-72eaa388-fa0b-4ce8-8695-6b1901c05616 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmppqdjwf37/3d6edbcf86824f0da598a5191c6c3caa.delta 1073741824" returned: 0 in 0.047s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:03 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-72eaa388-fa0b-4ce8-8695-6b1901c05616 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Quiescing instance not available: QEMU guest agent is not enabled.
Apr 12 03:32:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-72eaa388-fa0b-4ce8-8695-6b1901c05616 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Checking UEFI support for host arch (x86_64) {{(pid=72081) supports_uefi /opt/stack/nova/nova/virt/libvirt/host.py:1722}}
Apr 12 03:32:03 user nova-compute[72081]: INFO nova.virt.libvirt.host [None req-72eaa388-fa0b-4ce8-8695-6b1901c05616 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] UEFI support detected
Apr 12 03:32:03 user nova-compute[72081]: DEBUG nova.network.neutron [req-4d0fd191-b47f-42e6-bc6e-5a63ca689498 req-287284fb-bc04-4c07-886e-7663796fd558 service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4d0fd191-b47f-42e6-bc6e-5a63ca689498 req-287284fb-bc04-4c07-886e-7663796fd558 service nova] Releasing lock "refresh_cache-f728398d-524b-4f22-b91b-a431d62a952c" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Acquired lock "refresh_cache-f728398d-524b-4f22-b91b-a431d62a952c" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG nova.network.neutron [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG nova.network.neutron [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG nova.network.neutron [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Updating instance_info_cache with network_info: [{"id": "b0727a17-1b28-4bd2-a4bd-ba6f779fb19b", "address": "fa:16:3e:80:75:77", "network": {"id": "1a483654-b740-4544-84c9-53a41c11fa88", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1141588183-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "83a7ef3f97fd47849973d4e4482fb5f9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb0727a17-1b", "ovs_interfaceid": "b0727a17-1b28-4bd2-a4bd-ba6f779fb19b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-2d0904c3-74e0-47bb-a832-758a7035b2f5" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-2d0904c3-74e0-47bb-a832-758a7035b2f5" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:32:03 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Releasing lock "refresh_cache-f728398d-524b-4f22-b91b-a431d62a952c" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.compute.manager [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Instance network_info: |[{"id": "b0727a17-1b28-4bd2-a4bd-ba6f779fb19b", "address": "fa:16:3e:80:75:77", "network": {"id": "1a483654-b740-4544-84c9-53a41c11fa88", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1141588183-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "83a7ef3f97fd47849973d4e4482fb5f9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb0727a17-1b", "ovs_interfaceid": "b0727a17-1b28-4bd2-a4bd-ba6f779fb19b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Start _get_guest_xml network_info=[{"id": "b0727a17-1b28-4bd2-a4bd-ba6f779fb19b", "address": "fa:16:3e:80:75:77", "network": {"id": "1a483654-b740-4544-84c9-53a41c11fa88", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1141588183-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "83a7ef3f97fd47849973d4e4482fb5f9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb0727a17-1b", "ovs_interfaceid": "b0727a17-1b28-4bd2-a4bd-ba6f779fb19b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:32:04 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:32:04 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:32:00Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-1673161590',display_name='tempest-ServersNegativeTestJSON-server-1673161590',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-1673161590',id=12,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='83a7ef3f97fd47849973d4e4482fb5f9',ramdisk_id='',reservation_id='r-yad7o0xs',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServersNegativeTestJSON-864113759',owner_user_name='tempest-ServersNegativeTestJSON-864113759-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:32:01Z,user_data=None,user_id='3da7394c51664a7caacfda6b7dda9066',uuid=f728398d-524b-4f22-b91b-a431d62a952c,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "b0727a17-1b28-4bd2-a4bd-ba6f779fb19b", "address": "fa:16:3e:80:75:77", "network": {"id": "1a483654-b740-4544-84c9-53a41c11fa88", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1141588183-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "83a7ef3f97fd47849973d4e4482fb5f9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb0727a17-1b", "ovs_interfaceid": "b0727a17-1b28-4bd2-a4bd-ba6f779fb19b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Converting VIF {"id": "b0727a17-1b28-4bd2-a4bd-ba6f779fb19b", "address": "fa:16:3e:80:75:77", "network": {"id": "1a483654-b740-4544-84c9-53a41c11fa88", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1141588183-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "83a7ef3f97fd47849973d4e4482fb5f9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb0727a17-1b", "ovs_interfaceid": "b0727a17-1b28-4bd2-a4bd-ba6f779fb19b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:80:75:77,bridge_name='br-int',has_traffic_filtering=True,id=b0727a17-1b28-4bd2-a4bd-ba6f779fb19b,network=Network(1a483654-b740-4544-84c9-53a41c11fa88),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb0727a17-1b') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.objects.instance [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lazy-loading 'pci_devices' on Instance uuid f728398d-524b-4f22-b91b-a431d62a952c {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:32:04 user nova-compute[72081]:   <uuid>f728398d-524b-4f22-b91b-a431d62a952c</uuid>
Apr 12 03:32:04 user nova-compute[72081]:   <name>instance-0000000c</name>
Apr 12 03:32:04 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:32:04 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:32:04 user nova-compute[72081]:   <metadata>
Apr 12 03:32:04 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:32:04 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:32:04 user nova-compute[72081]:       <nova:name>tempest-ServersNegativeTestJSON-server-1673161590</nova:name>
Apr 12 03:32:04 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:32:04</nova:creationTime>
Apr 12 03:32:04 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:32:04 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:32:04 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:32:04 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:32:04 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:32:04 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:32:04 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:32:04 user nova-compute[72081]:       <nova:owner>
Apr 12 03:32:04 user nova-compute[72081]:         <nova:user uuid="3da7394c51664a7caacfda6b7dda9066">tempest-ServersNegativeTestJSON-864113759-project-member</nova:user>
Apr 12 03:32:04 user nova-compute[72081]:         <nova:project uuid="83a7ef3f97fd47849973d4e4482fb5f9">tempest-ServersNegativeTestJSON-864113759</nova:project>
Apr 12 03:32:04 user nova-compute[72081]:       </nova:owner>
Apr 12 03:32:04 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:32:04 user nova-compute[72081]:       <nova:ports>
Apr 12 03:32:04 user nova-compute[72081]:         <nova:port uuid="b0727a17-1b28-4bd2-a4bd-ba6f779fb19b">
Apr 12 03:32:04 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Apr 12 03:32:04 user nova-compute[72081]:         </nova:port>
Apr 12 03:32:04 user nova-compute[72081]:       </nova:ports>
Apr 12 03:32:04 user nova-compute[72081]:     </nova:instance>
Apr 12 03:32:04 user nova-compute[72081]:   </metadata>
Apr 12 03:32:04 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:32:04 user nova-compute[72081]:     <system>
Apr 12 03:32:04 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:32:04 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:32:04 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:32:04 user nova-compute[72081]:       <entry name="serial">f728398d-524b-4f22-b91b-a431d62a952c</entry>
Apr 12 03:32:04 user nova-compute[72081]:       <entry name="uuid">f728398d-524b-4f22-b91b-a431d62a952c</entry>
Apr 12 03:32:04 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:32:04 user nova-compute[72081]:     </system>
Apr 12 03:32:04 user nova-compute[72081]:   </sysinfo>
Apr 12 03:32:04 user nova-compute[72081]:   <os>
Apr 12 03:32:04 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:32:04 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:32:04 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:32:04 user nova-compute[72081]:   </os>
Apr 12 03:32:04 user nova-compute[72081]:   <features>
Apr 12 03:32:04 user nova-compute[72081]:     <acpi/>
Apr 12 03:32:04 user nova-compute[72081]:     <apic/>
Apr 12 03:32:04 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:32:04 user nova-compute[72081]:   </features>
Apr 12 03:32:04 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:32:04 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:32:04 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:32:04 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:32:04 user nova-compute[72081]:   </clock>
Apr 12 03:32:04 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:32:04 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:32:04 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:32:04 user nova-compute[72081]:   </cpu>
Apr 12 03:32:04 user nova-compute[72081]:   <devices>
Apr 12 03:32:04 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:32:04 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:32:04 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/f728398d-524b-4f22-b91b-a431d62a952c/disk"/>
Apr 12 03:32:04 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:32:04 user nova-compute[72081]:     </disk>
Apr 12 03:32:04 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:32:04 user nova-compute[72081]:       <mac address="fa:16:3e:80:75:77"/>
Apr 12 03:32:04 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:32:04 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:32:04 user nova-compute[72081]:       <target dev="tapb0727a17-1b"/>
Apr 12 03:32:04 user nova-compute[72081]:     </interface>
Apr 12 03:32:04 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:32:04 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/f728398d-524b-4f22-b91b-a431d62a952c/console.log" append="off"/>
Apr 12 03:32:04 user nova-compute[72081]:     </serial>
Apr 12 03:32:04 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:32:04 user nova-compute[72081]:     <video>
Apr 12 03:32:04 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:32:04 user nova-compute[72081]:     </video>
Apr 12 03:32:04 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:32:04 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:32:04 user nova-compute[72081]:     </rng>
Apr 12 03:32:04 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:32:04 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:32:04 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:32:04 user nova-compute[72081]:     </memballoon>
Apr 12 03:32:04 user nova-compute[72081]:   </devices>
Apr 12 03:32:04 user nova-compute[72081]: </domain>
Apr 12 03:32:04 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:32:00Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-1673161590',display_name='tempest-ServersNegativeTestJSON-server-1673161590',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-1673161590',id=12,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='83a7ef3f97fd47849973d4e4482fb5f9',ramdisk_id='',reservation_id='r-yad7o0xs',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServersNegativeTestJSON-864113759',owner_user_name='tempest-ServersNegativeTestJSON-864113759-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:32:01Z,user_data=None,user_id='3da7394c51664a7caacfda6b7dda9066',uuid=f728398d-524b-4f22-b91b-a431d62a952c,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "b0727a17-1b28-4bd2-a4bd-ba6f779fb19b", "address": "fa:16:3e:80:75:77", "network": {"id": "1a483654-b740-4544-84c9-53a41c11fa88", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1141588183-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "83a7ef3f97fd47849973d4e4482fb5f9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb0727a17-1b", "ovs_interfaceid": "b0727a17-1b28-4bd2-a4bd-ba6f779fb19b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Converting VIF {"id": "b0727a17-1b28-4bd2-a4bd-ba6f779fb19b", "address": "fa:16:3e:80:75:77", "network": {"id": "1a483654-b740-4544-84c9-53a41c11fa88", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1141588183-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "83a7ef3f97fd47849973d4e4482fb5f9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb0727a17-1b", "ovs_interfaceid": "b0727a17-1b28-4bd2-a4bd-ba6f779fb19b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:80:75:77,bridge_name='br-int',has_traffic_filtering=True,id=b0727a17-1b28-4bd2-a4bd-ba6f779fb19b,network=Network(1a483654-b740-4544-84c9-53a41c11fa88),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb0727a17-1b') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG os_vif [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:80:75:77,bridge_name='br-int',has_traffic_filtering=True,id=b0727a17-1b28-4bd2-a4bd-ba6f779fb19b,network=Network(1a483654-b740-4544-84c9-53a41c11fa88),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb0727a17-1b') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapb0727a17-1b, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapb0727a17-1b, col_values=(('external_ids', {'iface-id': 'b0727a17-1b28-4bd2-a4bd-ba6f779fb19b', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:80:75:77', 'vm-uuid': 'f728398d-524b-4f22-b91b-a431d62a952c'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:04 user nova-compute[72081]: INFO os_vif [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:80:75:77,bridge_name='br-int',has_traffic_filtering=True,id=b0727a17-1b28-4bd2-a4bd-ba6f779fb19b,network=Network(1a483654-b740-4544-84c9-53a41c11fa88),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb0727a17-1b')
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] No VIF found with MAC fa:16:3e:80:75:77, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-72eaa388-fa0b-4ce8-8695-6b1901c05616 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] COPY block job progress, current cursor: 0 final cursor: 43778048 {{(pid=72081) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32" by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32" acquired by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:04 user nova-compute[72081]: INFO nova.compute.manager [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Shelving
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-72eaa388-fa0b-4ce8-8695-6b1901c05616 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] COPY block job progress, current cursor: 43778048 final cursor: 43778048 {{(pid=72081) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 12 03:32:04 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-72eaa388-fa0b-4ce8-8695-6b1901c05616 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Skipping quiescing instance: QEMU guest agent is not enabled.
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Updating instance_info_cache with network_info: [{"id": "dce84390-a2f8-4047-b513-eec5845a2399", "address": "fa:16:3e:6e:2e:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdce84390-a2", "ovs_interfaceid": "dce84390-a2f8-4047-b513-eec5845a2399", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.privsep.utils [None req-72eaa388-fa0b-4ce8-8695-6b1901c05616 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72081) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-72eaa388-fa0b-4ce8-8695-6b1901c05616 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmppqdjwf37/3d6edbcf86824f0da598a5191c6c3caa.delta /opt/stack/data/nova/instances/snapshots/tmppqdjwf37/3d6edbcf86824f0da598a5191c6c3caa {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-2d0904c3-74e0-47bb-a832-758a7035b2f5" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:04 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-72eaa388-fa0b-4ce8-8695-6b1901c05616 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmppqdjwf37/3d6edbcf86824f0da598a5191c6c3caa.delta /opt/stack/data/nova/instances/snapshots/tmppqdjwf37/3d6edbcf86824f0da598a5191c6c3caa" returned: 0 in 0.225s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:05 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-72eaa388-fa0b-4ce8-8695-6b1901c05616 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Snapshot extracted, beginning image upload
Apr 12 03:32:05 user nova-compute[72081]: DEBUG nova.compute.manager [req-1dc2f8c3-d23a-454e-a6dd-da78e726814c req-35a5369d-6e9a-4245-9202-40d075c3fcce service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Received event network-vif-unplugged-540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1dc2f8c3-d23a-454e-a6dd-da78e726814c req-35a5369d-6e9a-4245-9202-40d075c3fcce service nova] Acquiring lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1dc2f8c3-d23a-454e-a6dd-da78e726814c req-35a5369d-6e9a-4245-9202-40d075c3fcce service nova] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1dc2f8c3-d23a-454e-a6dd-da78e726814c req-35a5369d-6e9a-4245-9202-40d075c3fcce service nova] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG nova.compute.manager [req-1dc2f8c3-d23a-454e-a6dd-da78e726814c req-35a5369d-6e9a-4245-9202-40d075c3fcce service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] No waiting events found dispatching network-vif-unplugged-540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:05 user nova-compute[72081]: WARNING nova.compute.manager [req-1dc2f8c3-d23a-454e-a6dd-da78e726814c req-35a5369d-6e9a-4245-9202-40d075c3fcce service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Received unexpected event network-vif-unplugged-540b7307-8fea-4b50-bab8-626983135a96 for instance with vm_state active and task_state shelving.
Apr 12 03:32:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Acquiring lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG nova.objects.instance [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lazy-loading 'flavor' on Instance uuid ef4ab4cb-6164-47ea-be1d-cbe0698f991d {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.112s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Acquiring lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:05 user nova-compute[72081]: INFO nova.compute.manager [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Attaching volume 23b341e1-3bd9-452a-851e-83de16a594e6 to /dev/vdb
Apr 12 03:32:05 user nova-compute[72081]: DEBUG os_brick.utils [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d75ec687-1349-4783-af62-be97974b6529]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8e4700c4-443c-444b-948f-43cc37a30ff9]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[fa8f7354-634e-4730-b647-7ec4a57a3620]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0edbf748-2989-40a8-b987-c90adddfecab]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:05 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Instance destroyed successfully.
Apr 12 03:32:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:32:05 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:32:05 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2d18d694-01f5-433c-8d90-52d2026571d2]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG os_brick.utils [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] <== get_connector_properties: return (151ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Updating existing volume attachment record: 9ab89fd4-bf13-4609-b463-f4169f197ff1 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:32:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:06 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Beginning cold snapshot process
Apr 12 03:32:06 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:06 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:06 user nova-compute[72081]: DEBUG nova.privsep.utils [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72081) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 12 03:32:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk /opt/stack/data/nova/instances/snapshots/tmpi__nrhi_/426e384525744cdca432b40357e35916 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk /opt/stack/data/nova/instances/snapshots/tmpi__nrhi_/426e384525744cdca432b40357e35916" returned: 0 in 0.512s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:06 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Snapshot extracted, beginning image upload
Apr 12 03:32:06 user nova-compute[72081]: DEBUG nova.compute.manager [req-70cfc7e3-8b03-433d-87f6-5d367fca5719 req-7914faf0-eba1-42be-9d83-bda9c35e094f service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received event network-changed-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:06 user nova-compute[72081]: DEBUG nova.compute.manager [req-70cfc7e3-8b03-433d-87f6-5d367fca5719 req-7914faf0-eba1-42be-9d83-bda9c35e094f service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Refreshing instance network info cache due to event network-changed-cff588d4-f46a-4d44-a3e6-1c04f6abd380. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:32:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-70cfc7e3-8b03-433d-87f6-5d367fca5719 req-7914faf0-eba1-42be-9d83-bda9c35e094f service nova] Acquiring lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:32:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-70cfc7e3-8b03-433d-87f6-5d367fca5719 req-7914faf0-eba1-42be-9d83-bda9c35e094f service nova] Acquired lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:32:06 user nova-compute[72081]: DEBUG nova.network.neutron [req-70cfc7e3-8b03-433d-87f6-5d367fca5719 req-7914faf0-eba1-42be-9d83-bda9c35e094f service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Refreshing network info cache for port cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:32:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-2bddd0ed-55e1-405f-a106-70cd6f546cac req-ac0b5f2f-e2bc-47c9-b38c-26c555e9371a service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Received event network-vif-plugged-540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2bddd0ed-55e1-405f-a106-70cd6f546cac req-ac0b5f2f-e2bc-47c9-b38c-26c555e9371a service nova] Acquiring lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2bddd0ed-55e1-405f-a106-70cd6f546cac req-ac0b5f2f-e2bc-47c9-b38c-26c555e9371a service nova] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2bddd0ed-55e1-405f-a106-70cd6f546cac req-ac0b5f2f-e2bc-47c9-b38c-26c555e9371a service nova] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-2bddd0ed-55e1-405f-a106-70cd6f546cac req-ac0b5f2f-e2bc-47c9-b38c-26c555e9371a service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] No waiting events found dispatching network-vif-plugged-540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:07 user nova-compute[72081]: WARNING nova.compute.manager [req-2bddd0ed-55e1-405f-a106-70cd6f546cac req-ac0b5f2f-e2bc-47c9-b38c-26c555e9371a service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Received unexpected event network-vif-plugged-540b7307-8fea-4b50-bab8-626983135a96 for instance with vm_state active and task_state shelving_image_uploading.
Apr 12 03:32:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-2bddd0ed-55e1-405f-a106-70cd6f546cac req-ac0b5f2f-e2bc-47c9-b38c-26c555e9371a service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Received event network-vif-plugged-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2bddd0ed-55e1-405f-a106-70cd6f546cac req-ac0b5f2f-e2bc-47c9-b38c-26c555e9371a service nova] Acquiring lock "f728398d-524b-4f22-b91b-a431d62a952c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2bddd0ed-55e1-405f-a106-70cd6f546cac req-ac0b5f2f-e2bc-47c9-b38c-26c555e9371a service nova] Lock "f728398d-524b-4f22-b91b-a431d62a952c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2bddd0ed-55e1-405f-a106-70cd6f546cac req-ac0b5f2f-e2bc-47c9-b38c-26c555e9371a service nova] Lock "f728398d-524b-4f22-b91b-a431d62a952c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-2bddd0ed-55e1-405f-a106-70cd6f546cac req-ac0b5f2f-e2bc-47c9-b38c-26c555e9371a service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] No waiting events found dispatching network-vif-plugged-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:07 user nova-compute[72081]: WARNING nova.compute.manager [req-2bddd0ed-55e1-405f-a106-70cd6f546cac req-ac0b5f2f-e2bc-47c9-b38c-26c555e9371a service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Received unexpected event network-vif-plugged-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b for instance with vm_state building and task_state spawning.
Apr 12 03:32:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-2bddd0ed-55e1-405f-a106-70cd6f546cac req-ac0b5f2f-e2bc-47c9-b38c-26c555e9371a service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Received event network-vif-plugged-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2bddd0ed-55e1-405f-a106-70cd6f546cac req-ac0b5f2f-e2bc-47c9-b38c-26c555e9371a service nova] Acquiring lock "f728398d-524b-4f22-b91b-a431d62a952c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2bddd0ed-55e1-405f-a106-70cd6f546cac req-ac0b5f2f-e2bc-47c9-b38c-26c555e9371a service nova] Lock "f728398d-524b-4f22-b91b-a431d62a952c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2bddd0ed-55e1-405f-a106-70cd6f546cac req-ac0b5f2f-e2bc-47c9-b38c-26c555e9371a service nova] Lock "f728398d-524b-4f22-b91b-a431d62a952c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-2bddd0ed-55e1-405f-a106-70cd6f546cac req-ac0b5f2f-e2bc-47c9-b38c-26c555e9371a service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] No waiting events found dispatching network-vif-plugged-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:07 user nova-compute[72081]: WARNING nova.compute.manager [req-2bddd0ed-55e1-405f-a106-70cd6f546cac req-ac0b5f2f-e2bc-47c9-b38c-26c555e9371a service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Received unexpected event network-vif-plugged-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b for instance with vm_state building and task_state spawning.
Apr 12 03:32:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:07 user nova-compute[72081]: DEBUG nova.network.neutron [req-70cfc7e3-8b03-433d-87f6-5d367fca5719 req-7914faf0-eba1-42be-9d83-bda9c35e094f service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Updated VIF entry in instance network info cache for port cff588d4-f46a-4d44-a3e6-1c04f6abd380. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:32:07 user nova-compute[72081]: DEBUG nova.network.neutron [req-70cfc7e3-8b03-433d-87f6-5d367fca5719 req-7914faf0-eba1-42be-9d83-bda9c35e094f service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Updating instance_info_cache with network_info: [{"id": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "address": "fa:16:3e:c0:f6:ee", "network": {"id": "b4464ad0-6022-43e7-bfe4-229e6a3a0600", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1704232678-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.188", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a027d7708e6747fe970c388d6ba66359", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcff588d4-f4", "ovs_interfaceid": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:32:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-70cfc7e3-8b03-433d-87f6-5d367fca5719 req-7914faf0-eba1-42be-9d83-bda9c35e094f service nova] Releasing lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:32:07 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-72eaa388-fa0b-4ce8-8695-6b1901c05616 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Snapshot image upload complete
Apr 12 03:32:07 user nova-compute[72081]: INFO nova.compute.manager [None req-72eaa388-fa0b-4ce8-8695-6b1901c05616 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Took 5.06 seconds to snapshot the instance on the hypervisor.
Apr 12 03:32:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-5bc25c7e-7e44-4cf1-b19a-7e3475757df6 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:32:07 user nova-compute[72081]: INFO nova.compute.manager [None req-5bc25c7e-7e44-4cf1-b19a-7e3475757df6 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] instance snapshotting
Apr 12 03:32:07 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-5bc25c7e-7e44-4cf1-b19a-7e3475757df6 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Beginning live snapshot process
Apr 12 03:32:08 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270328.0548968, f728398d-524b-4f22-b91b-a431d62a952c => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:32:08 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f728398d-524b-4f22-b91b-a431d62a952c] VM Resumed (Lifecycle Event)
Apr 12 03:32:08 user nova-compute[72081]: DEBUG nova.compute.manager [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:32:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:32:08 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Instance spawned successfully.
Apr 12 03:32:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:32:08 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:32:08 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:32:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:32:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:32:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:32:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:32:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:32:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:32:08 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f728398d-524b-4f22-b91b-a431d62a952c] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:32:08 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270328.0603101, f728398d-524b-4f22-b91b-a431d62a952c => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:32:08 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f728398d-524b-4f22-b91b-a431d62a952c] VM Started (Lifecycle Event)
Apr 12 03:32:08 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:32:08 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:32:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-5bc25c7e-7e44-4cf1-b19a-7e3475757df6 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json -f qcow2 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:08 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f728398d-524b-4f22-b91b-a431d62a952c] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:32:08 user nova-compute[72081]: INFO nova.compute.manager [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Took 6.12 seconds to spawn the instance on the hypervisor.
Apr 12 03:32:08 user nova-compute[72081]: DEBUG nova.compute.manager [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:32:08 user nova-compute[72081]: INFO nova.compute.manager [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Took 7.66 seconds to build instance.
Apr 12 03:32:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-65154dff-aed8-4e8e-aa78-24272b5bbaa3 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "f728398d-524b-4f22-b91b-a431d62a952c" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.775s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-5bc25c7e-7e44-4cf1-b19a-7e3475757df6 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json -f qcow2" returned: 0 in 0.152s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-5bc25c7e-7e44-4cf1-b19a-7e3475757df6 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json -f qcow2 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-5bc25c7e-7e44-4cf1-b19a-7e3475757df6 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json -f qcow2" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-5bc25c7e-7e44-4cf1-b19a-7e3475757df6 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-5bc25c7e-7e44-4cf1-b19a-7e3475757df6 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.143s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-5bc25c7e-7e44-4cf1-b19a-7e3475757df6 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmpgxl9ap83/d2aad2dbd2cc49b0a7c79be3915a19cd.delta 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-5bc25c7e-7e44-4cf1-b19a-7e3475757df6 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmpgxl9ap83/d2aad2dbd2cc49b0a7c79be3915a19cd.delta 1073741824" returned: 0 in 0.052s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:08 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-5bc25c7e-7e44-4cf1-b19a-7e3475757df6 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Quiescing instance not available: QEMU guest agent is not enabled.
Apr 12 03:32:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:09 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Snapshot image upload complete
Apr 12 03:32:09 user nova-compute[72081]: DEBUG nova.compute.manager [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:32:09 user nova-compute[72081]: INFO nova.compute.manager [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Shelve offloading
Apr 12 03:32:09 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Instance destroyed successfully.
Apr 12 03:32:09 user nova-compute[72081]: DEBUG nova.compute.manager [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:32:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "refresh_cache-b23d4bcd-989a-45e1-89a3-6e3d30229a32" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:32:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquired lock "refresh_cache-b23d4bcd-989a-45e1-89a3-6e3d30229a32" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:32:09 user nova-compute[72081]: DEBUG nova.network.neutron [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:32:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-5bc25c7e-7e44-4cf1-b19a-7e3475757df6 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] COPY block job progress, current cursor: 0 final cursor: 43778048 {{(pid=72081) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 12 03:32:09 user nova-compute[72081]: DEBUG nova.network.neutron [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Updating instance_info_cache with network_info: [{"id": "540b7307-8fea-4b50-bab8-626983135a96", "address": "fa:16:3e:8d:a9:ab", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.135", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap540b7307-8f", "ovs_interfaceid": "540b7307-8fea-4b50-bab8-626983135a96", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:32:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Releasing lock "refresh_cache-b23d4bcd-989a-45e1-89a3-6e3d30229a32" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-5bc25c7e-7e44-4cf1-b19a-7e3475757df6 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] COPY block job progress, current cursor: 43778048 final cursor: 43778048 {{(pid=72081) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 12 03:32:10 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-5bc25c7e-7e44-4cf1-b19a-7e3475757df6 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Skipping quiescing instance: QEMU guest agent is not enabled.
Apr 12 03:32:10 user nova-compute[72081]: DEBUG nova.privsep.utils [None req-5bc25c7e-7e44-4cf1-b19a-7e3475757df6 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72081) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-5bc25c7e-7e44-4cf1-b19a-7e3475757df6 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmpgxl9ap83/d2aad2dbd2cc49b0a7c79be3915a19cd.delta /opt/stack/data/nova/instances/snapshots/tmpgxl9ap83/d2aad2dbd2cc49b0a7c79be3915a19cd {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-5bc25c7e-7e44-4cf1-b19a-7e3475757df6 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmpgxl9ap83/d2aad2dbd2cc49b0a7c79be3915a19cd.delta /opt/stack/data/nova/instances/snapshots/tmpgxl9ap83/d2aad2dbd2cc49b0a7c79be3915a19cd" returned: 0 in 0.181s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:10 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-5bc25c7e-7e44-4cf1-b19a-7e3475757df6 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Snapshot extracted, beginning image upload
Apr 12 03:32:10 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Instance destroyed successfully.
Apr 12 03:32:10 user nova-compute[72081]: DEBUG nova.objects.instance [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lazy-loading 'resources' on Instance uuid b23d4bcd-989a-45e1-89a3-6e3d30229a32 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:30:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-1925048093',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-1925048093',id=3,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBNv3aMnfqZBrraTWMhLhVcBPnIqd/pyo9633J0szzOpRjklWBRcLk+TXv631ENNLeAHi6cKzIvn2qCEpBSN9HX63yWj5klT0kBwkXMpaNwu0eHxsmOThEKjvkK+5wRQIg==',key_name='tempest-keypair-892402082',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:30:30Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='738ebb6b798c4974aa46f9bbe3828279',ramdisk_id='',reservation_id='r-ybseqfps',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-2139975165',owner_user_name='tempest-AttachVolumeShelveTestJSON-2139975165-project-member',shelved_at='2023-04-12T03:32:09.253612',shelved_host='user',shelved_image_id='1feda9ce-2630-410a-acd7-d4d319edb2af'},tags=<?>,task_state='shelving_offloading',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:32:07Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='de1fe5db01624224ab7b615c908d990b',uuid=b23d4bcd-989a-45e1-89a3-6e3d30229a32,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='shelved') vif={"id": "540b7307-8fea-4b50-bab8-626983135a96", "address": "fa:16:3e:8d:a9:ab", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.135", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap540b7307-8f", "ovs_interfaceid": "540b7307-8fea-4b50-bab8-626983135a96", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Converting VIF {"id": "540b7307-8fea-4b50-bab8-626983135a96", "address": "fa:16:3e:8d:a9:ab", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.135", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap540b7307-8f", "ovs_interfaceid": "540b7307-8fea-4b50-bab8-626983135a96", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:8d:a9:ab,bridge_name='br-int',has_traffic_filtering=True,id=540b7307-8fea-4b50-bab8-626983135a96,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap540b7307-8f') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG os_vif [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:8d:a9:ab,bridge_name='br-int',has_traffic_filtering=True,id=540b7307-8fea-4b50-bab8-626983135a96,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap540b7307-8f') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap540b7307-8f, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:10 user nova-compute[72081]: INFO os_vif [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:8d:a9:ab,bridge_name='br-int',has_traffic_filtering=True,id=540b7307-8fea-4b50-bab8-626983135a96,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap540b7307-8f')
Apr 12 03:32:10 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Deleting instance files /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32_del
Apr 12 03:32:10 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Deletion of /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32_del complete
Apr 12 03:32:10 user nova-compute[72081]: DEBUG nova.compute.manager [req-7928f17f-209b-4ee8-a4c4-9a3e34da17c9 req-6bc45eb6-f9ac-416c-9a20-8bed8d17439b service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Received event network-changed-540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG nova.compute.manager [req-7928f17f-209b-4ee8-a4c4-9a3e34da17c9 req-6bc45eb6-f9ac-416c-9a20-8bed8d17439b service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Refreshing instance network info cache due to event network-changed-540b7307-8fea-4b50-bab8-626983135a96. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7928f17f-209b-4ee8-a4c4-9a3e34da17c9 req-6bc45eb6-f9ac-416c-9a20-8bed8d17439b service nova] Acquiring lock "refresh_cache-b23d4bcd-989a-45e1-89a3-6e3d30229a32" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7928f17f-209b-4ee8-a4c4-9a3e34da17c9 req-6bc45eb6-f9ac-416c-9a20-8bed8d17439b service nova] Acquired lock "refresh_cache-b23d4bcd-989a-45e1-89a3-6e3d30229a32" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG nova.network.neutron [req-7928f17f-209b-4ee8-a4c4-9a3e34da17c9 req-6bc45eb6-f9ac-416c-9a20-8bed8d17439b service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Refreshing network info cache for port 540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OFJjNmWs', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:32:10 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:32:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:32:10 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:32:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:32:10 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:32:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 -p 172.16.0.220:3260" returned: 21 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5567aabd-aca7-4db5-b68a-800fb3d3593b]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:32:10 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[75f9c8c7-fccf-4e58-bb10-bcd5bb9020c0]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6] added
Apr 12 03:32:10 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[eeee6fa0-d68b-49bd-9624-6ab5a85ff51c]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:10 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Deleted allocations for instance b23d4bcd-989a-45e1-89a3-6e3d30229a32
Apr 12 03:32:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6ac657fe-9a17-45ac-a440-d10258fd0911]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v OFJjNmWs {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v OFJjNmWs" returned: 0 in 0.084s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c259378a-f532-4e3a-a283-775acc9454d9]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'OFJjNmWs'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG nova.objects.instance [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lazy-loading 'numa_topology' on Instance uuid b23d4bcd-989a-45e1-89a3-6e3d30229a32 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ea7680d7-d547-44d6-8a7d-c3b97bae2ae8]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ed98929d-cf31-4c66-b53e-6b431a5ed941]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:11 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:11 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:11 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:11 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:11 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:11 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:11 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:11 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:11 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:11 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:11 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:11 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 -p 172.16.0.220:3260 --login" returned: 0 in 0.082s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[014f33de-c917-401a-958f-2c2586f16b2a]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6, portal: 172.16.0.220,3260]
Apr 12 03:32:11 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6, portal: 172.16.0.220,3260] successful.
Apr 12 03:32:11 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0e447628-080c-43b5-aa81-ebe6c5d9623d]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.031s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[901ecb7c-3a07-4dd5-8daa-b67d8f140773]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:11 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:11 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:11 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:11 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:11 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:11 user nova-compute[72081]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 (non-flash)
Apr 12 03:32:11 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:11 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:11 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:11 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:11 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:11 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:11 user nova-compute[72081]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 (non-flash)
Apr 12 03:32:11 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.434s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] HCTL ('39', '-', '-', 0) found on session 8 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Scanning host 39 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host39/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host39/scan" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[09d8c4cc-3a70-47dd-a121-8e9e4e60fdf1]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Searching for a device in session 8 and hctl ['39', '*', '*', 0] yield: sdh {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Connected to sdh using {'target_iqn': 'iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OFJjNmWs', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.449s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG nova.network.neutron [req-7928f17f-209b-4ee8-a4c4-9a3e34da17c9 req-6bc45eb6-f9ac-416c-9a20-8bed8d17439b service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Updated VIF entry in instance network info cache for port 540b7307-8fea-4b50-bab8-626983135a96. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG nova.network.neutron [req-7928f17f-209b-4ee8-a4c4-9a3e34da17c9 req-6bc45eb6-f9ac-416c-9a20-8bed8d17439b service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Updating instance_info_cache with network_info: [{"id": "540b7307-8fea-4b50-bab8-626983135a96", "address": "fa:16:3e:8d:a9:ab", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": null, "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.135", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "unbound", "details": {}, "devname": "tap540b7307-8f", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7928f17f-209b-4ee8-a4c4-9a3e34da17c9 req-6bc45eb6-f9ac-416c-9a20-8bed8d17439b service nova] Releasing lock "refresh_cache-b23d4bcd-989a-45e1-89a3-6e3d30229a32" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:32:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-073bc7f5-3172-4aaa-82c2-c67395afd7ba tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32" "released" by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" :: held 6.727s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:12 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.479s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:32:12 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] <== connect_volume: return (1481ms) {'type': 'block', 'scsi_wwn': '26236366237373134', 'path': '/dev/sdh'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:32:12 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26236366237373134', 'path': '/dev/sdh'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:32:12 user nova-compute[72081]: DEBUG nova.objects.instance [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lazy-loading 'flavor' on Instance uuid ef4ab4cb-6164-47ea-be1d-cbe0698f991d {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:32:12 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] attach device xml: <disk type="block" device="disk">
Apr 12 03:32:12 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:32:12 user nova-compute[72081]:   <source dev="/dev/sdh"/>
Apr 12 03:32:12 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:32:12 user nova-compute[72081]:   <serial>23b341e1-3bd9-452a-851e-83de16a594e6</serial>
Apr 12 03:32:12 user nova-compute[72081]: </disk>
Apr 12 03:32:12 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 03:32:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:12 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-5bc25c7e-7e44-4cf1-b19a-7e3475757df6 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Snapshot image upload complete
Apr 12 03:32:12 user nova-compute[72081]: INFO nova.compute.manager [None req-5bc25c7e-7e44-4cf1-b19a-7e3475757df6 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Took 4.72 seconds to snapshot the instance on the hypervisor.
Apr 12 03:32:12 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:32:12 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:32:12 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] No VIF found with MAC fa:16:3e:ac:8c:49, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:32:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-04712b1c-dae6-42c8-8157-6abf69bcc31d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 7.110s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Acquiring lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Acquiring lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:13 user nova-compute[72081]: INFO nova.compute.manager [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Terminating instance
Apr 12 03:32:13 user nova-compute[72081]: DEBUG nova.compute.manager [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:32:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:13 user nova-compute[72081]: DEBUG nova.compute.manager [req-8bf98f4e-a844-4648-9aea-0a3916069d8b req-759f67ba-2112-4b06-80e1-8877a063e22f service nova] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Received event network-vif-unplugged-cc0a860c-4dc1-4a09-836a-3878bcb30d40 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8bf98f4e-a844-4648-9aea-0a3916069d8b req-759f67ba-2112-4b06-80e1-8877a063e22f service nova] Acquiring lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8bf98f4e-a844-4648-9aea-0a3916069d8b req-759f67ba-2112-4b06-80e1-8877a063e22f service nova] Lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8bf98f4e-a844-4648-9aea-0a3916069d8b req-759f67ba-2112-4b06-80e1-8877a063e22f service nova] Lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:13 user nova-compute[72081]: DEBUG nova.compute.manager [req-8bf98f4e-a844-4648-9aea-0a3916069d8b req-759f67ba-2112-4b06-80e1-8877a063e22f service nova] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] No waiting events found dispatching network-vif-unplugged-cc0a860c-4dc1-4a09-836a-3878bcb30d40 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:13 user nova-compute[72081]: DEBUG nova.compute.manager [req-8bf98f4e-a844-4648-9aea-0a3916069d8b req-759f67ba-2112-4b06-80e1-8877a063e22f service nova] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Received event network-vif-unplugged-cc0a860c-4dc1-4a09-836a-3878bcb30d40 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:32:14 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Instance destroyed successfully.
Apr 12 03:32:14 user nova-compute[72081]: DEBUG nova.objects.instance [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lazy-loading 'resources' on Instance uuid ef4ab4cb-6164-47ea-be1d-cbe0698f991d {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:30:13Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-DeleteServersTestJSON-server-160677355',display_name='tempest-DeleteServersTestJSON-server-160677355',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-deleteserverstestjson-server-160677355',id=4,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:30:32Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='5dadbf1b0a124d2d929c85bc2b916022',ramdisk_id='',reservation_id='r-auk5xva2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-DeleteServersTestJSON-78784513',owner_user_name='tempest-DeleteServersTestJSON-78784513-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:30:33Z,user_data=None,user_id='4d01e7aaa3734b299d0c0a27844011b4',uuid=ef4ab4cb-6164-47ea-be1d-cbe0698f991d,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "cc0a860c-4dc1-4a09-836a-3878bcb30d40", "address": "fa:16:3e:ac:8c:49", "network": {"id": "0f8e4d8d-8d90-47de-a965-5610331236ef", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-2097572913-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5dadbf1b0a124d2d929c85bc2b916022", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcc0a860c-4d", "ovs_interfaceid": "cc0a860c-4dc1-4a09-836a-3878bcb30d40", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Converting VIF {"id": "cc0a860c-4dc1-4a09-836a-3878bcb30d40", "address": "fa:16:3e:ac:8c:49", "network": {"id": "0f8e4d8d-8d90-47de-a965-5610331236ef", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-2097572913-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5dadbf1b0a124d2d929c85bc2b916022", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcc0a860c-4d", "ovs_interfaceid": "cc0a860c-4dc1-4a09-836a-3878bcb30d40", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ac:8c:49,bridge_name='br-int',has_traffic_filtering=True,id=cc0a860c-4dc1-4a09-836a-3878bcb30d40,network=Network(0f8e4d8d-8d90-47de-a965-5610331236ef),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapcc0a860c-4d') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG os_vif [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:ac:8c:49,bridge_name='br-int',has_traffic_filtering=True,id=cc0a860c-4dc1-4a09-836a-3878bcb30d40,network=Network(0f8e4d8d-8d90-47de-a965-5610331236ef),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapcc0a860c-4d') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapcc0a860c-4d, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:32:14 user nova-compute[72081]: INFO os_vif [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:ac:8c:49,bridge_name='br-int',has_traffic_filtering=True,id=cc0a860c-4dc1-4a09-836a-3878bcb30d40,network=Network(0f8e4d8d-8d90-47de-a965-5610331236ef),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapcc0a860c-4d')
Apr 12 03:32:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OFJjNmWs', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdh': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:32:14 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:32:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[23c0c31b-9d8e-491c-b1fb-c5b3e64465e8]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:32:14 user nova-compute[72081]: No targets found.
Apr 12 03:32:14 user nova-compute[72081]: iSNS:
Apr 12 03:32:14 user nova-compute[72081]: No targets found.
Apr 12 03:32:14 user nova-compute[72081]: STATIC:
Apr 12 03:32:14 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9
Apr 12 03:32:14 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:14 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:14 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1
Apr 12 03:32:14 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:14 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:14 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3
Apr 12 03:32:14 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:14 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:14 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d
Apr 12 03:32:14 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:14 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:14 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd
Apr 12 03:32:14 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:14 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:14 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352
Apr 12 03:32:14 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:14 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:14 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6
Apr 12 03:32:14 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:14 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:14 user nova-compute[72081]: FIRMWARE:
Apr 12 03:32:14 user nova-compute[72081]: No targets found.
Apr 12 03:32:14 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:32:14 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:32:14 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[169f8a09-239c-4e3b-9fdd-ef9d0301262a]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.040s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[67ac1eb5-7c61-4662-86aa-ded009c02245]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:14 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:14 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:14 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:14 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:14 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:14 user nova-compute[72081]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 (non-flash)
Apr 12 03:32:14 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:14 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:14 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:14 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:14 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:14 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:14 user nova-compute[72081]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 (non-flash)
Apr 12 03:32:14 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fb61af520>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6'): ({'sdh'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Removing single pathed devices sdh {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9401fff1-a2dc-4a0f-80c1-e1c4afe95cd7]: (4, ('path checker states:\nup                  7\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdh {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdh" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3cfc4fcc-ab72-4f34-bea7-60f9e8695ac2]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Flushing IO for device /dev/sdh {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdh {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdh" returned: 0 in 0.012s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d1cbded-1860-4955-a174-cbcc2f84cf21]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Remove SCSI device /dev/sdh with /sys/block/sdh/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdh/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdh/device/delete" returned: 0 in 0.040s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[20866631-af3e-48d2-a69e-29b728318cd7]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Checking to see if SCSI volumes sdh have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] SCSI volumes sdh have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6745191a-5682-4ea5-acb8-eb244c6bf084]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 -p 172.16.0.220:3260 --logout" returned: 0 in 0.064s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[029d05dd-71ec-4090-b9e5-fb588d9528ac]: (4, ('Logging out of session [sid: 8, target: iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6, portal: 172.16.0.220,3260]\nLogout of [sid: 8, target: iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 8, target: iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6, portal: 172.16.0.220,3260]
Apr 12 03:32:14 user nova-compute[72081]: Logout of [sid: 8, target: iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6, portal: 172.16.0.220,3260] successful.
Apr 12 03:32:14 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:23b341e1-3bd9-452a-851e-83de16a594e6 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.031s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7cf6c9d8-8527-45aa-8962-64106cda1ad6]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.330s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] <== disconnect_volume: return (331ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:32:14 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Deleting instance files /opt/stack/data/nova/instances/ef4ab4cb-6164-47ea-be1d-cbe0698f991d_del
Apr 12 03:32:14 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Deletion of /opt/stack/data/nova/instances/ef4ab4cb-6164-47ea-be1d-cbe0698f991d_del complete
Apr 12 03:32:14 user nova-compute[72081]: INFO nova.compute.manager [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Took 1.21 seconds to destroy the instance on the hypervisor.
Apr 12 03:32:14 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:32:14 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:32:15 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:32:15 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Took 0.59 seconds to deallocate network for instance.
Apr 12 03:32:15 user nova-compute[72081]: DEBUG nova.compute.manager [req-c249cfef-d3ff-4928-8189-5f7228f01026 req-063bab42-82d9-4009-97aa-54a555d19aed service nova] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Received event network-vif-deleted-cc0a860c-4dc1-4a09-836a-3878bcb30d40 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:15 user nova-compute[72081]: DEBUG nova.compute.manager [req-cc8e20d9-a45e-4ae6-8f21-67b410d7a117 req-513a7b13-fb61-4676-8f73-1088ccd73a7c service nova] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Received event network-vif-plugged-cc0a860c-4dc1-4a09-836a-3878bcb30d40 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc8e20d9-a45e-4ae6-8f21-67b410d7a117 req-513a7b13-fb61-4676-8f73-1088ccd73a7c service nova] Acquiring lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc8e20d9-a45e-4ae6-8f21-67b410d7a117 req-513a7b13-fb61-4676-8f73-1088ccd73a7c service nova] Lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc8e20d9-a45e-4ae6-8f21-67b410d7a117 req-513a7b13-fb61-4676-8f73-1088ccd73a7c service nova] Lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:15 user nova-compute[72081]: DEBUG nova.compute.manager [req-cc8e20d9-a45e-4ae6-8f21-67b410d7a117 req-513a7b13-fb61-4676-8f73-1088ccd73a7c service nova] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] No waiting events found dispatching network-vif-plugged-cc0a860c-4dc1-4a09-836a-3878bcb30d40 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:15 user nova-compute[72081]: WARNING nova.compute.manager [req-cc8e20d9-a45e-4ae6-8f21-67b410d7a117 req-513a7b13-fb61-4676-8f73-1088ccd73a7c service nova] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Received unexpected event network-vif-plugged-cc0a860c-4dc1-4a09-836a-3878bcb30d40 for instance with vm_state active and task_state deleting.
Apr 12 03:32:16 user nova-compute[72081]: INFO nova.compute.manager [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Took 0.80 seconds to detach 1 volumes for instance.
Apr 12 03:32:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:16 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:32:16 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:32:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.308s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:16 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Deleted allocations for instance ef4ab4cb-6164-47ea-be1d-cbe0698f991d
Apr 12 03:32:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-be494c85-a24e-4505-8d06-e562ed5c548d tempest-DeleteServersTestJSON-78784513 tempest-DeleteServersTestJSON-78784513-project-member] Lock "ef4ab4cb-6164-47ea-be1d-cbe0698f991d" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.103s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:20 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681270325.8324666, b23d4bcd-989a-45e1-89a3-6e3d30229a32 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:32:20 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] VM Stopped (Lifecycle Event)
Apr 12 03:32:20 user nova-compute[72081]: DEBUG nova.compute.manager [None req-7c4af607-1e48-422b-bdbf-975fa1880a8c None None] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:32:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:23 user nova-compute[72081]: DEBUG nova.compute.manager [req-e6a62604-eda8-4c09-b1a4-20e7eaf2d06b req-9f8d6d9c-631c-4342-9242-9112bc711a9f service nova] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Received event network-changed-b58865c9-a096-40af-91cf-828e5131ea22 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:23 user nova-compute[72081]: DEBUG nova.compute.manager [req-e6a62604-eda8-4c09-b1a4-20e7eaf2d06b req-9f8d6d9c-631c-4342-9242-9112bc711a9f service nova] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Refreshing instance network info cache due to event network-changed-b58865c9-a096-40af-91cf-828e5131ea22. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:32:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e6a62604-eda8-4c09-b1a4-20e7eaf2d06b req-9f8d6d9c-631c-4342-9242-9112bc711a9f service nova] Acquiring lock "refresh_cache-ccf9f424-d230-4079-87c5-bb08bc85b0b7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:32:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e6a62604-eda8-4c09-b1a4-20e7eaf2d06b req-9f8d6d9c-631c-4342-9242-9112bc711a9f service nova] Acquired lock "refresh_cache-ccf9f424-d230-4079-87c5-bb08bc85b0b7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:32:23 user nova-compute[72081]: DEBUG nova.network.neutron [req-e6a62604-eda8-4c09-b1a4-20e7eaf2d06b req-9f8d6d9c-631c-4342-9242-9112bc711a9f service nova] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Refreshing network info cache for port b58865c9-a096-40af-91cf-828e5131ea22 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:32:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Acquiring lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:23 user nova-compute[72081]: DEBUG nova.objects.instance [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lazy-loading 'flavor' on Instance uuid ccf9f424-d230-4079-87c5-bb08bc85b0b7 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:32:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.057s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Acquiring lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:23 user nova-compute[72081]: INFO nova.compute.manager [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Attaching volume 6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 to /dev/vdb
Apr 12 03:32:23 user nova-compute[72081]: DEBUG os_brick.utils [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:32:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:24 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[216c89fd-9c58-4c82-94b9-a2a8ce2ad939]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:24 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[623aef5e-5aab-4ea1-8a43-480662215e07]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:24 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8ecb9eb6-a450-4f84-980c-4e76b5d8e3bf]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:24 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1a2856a5-4e05-497a-8e9d-9d295d6a5347]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:32:24 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:32:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:32:24 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:32:24 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6e35e42d-860a-4559-8f47-b3ce508bd3f8]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:24 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:32:24 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:32:24 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:32:24 user nova-compute[72081]: DEBUG os_brick.utils [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] <== get_connector_properties: return (156ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:32:24 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Updating existing volume attachment record: 1ee892b5-f227-407d-a4bf-c4b23bafcbf6 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:32:24 user nova-compute[72081]: DEBUG nova.network.neutron [req-e6a62604-eda8-4c09-b1a4-20e7eaf2d06b req-9f8d6d9c-631c-4342-9242-9112bc711a9f service nova] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Updated VIF entry in instance network info cache for port b58865c9-a096-40af-91cf-828e5131ea22. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:32:24 user nova-compute[72081]: DEBUG nova.network.neutron [req-e6a62604-eda8-4c09-b1a4-20e7eaf2d06b req-9f8d6d9c-631c-4342-9242-9112bc711a9f service nova] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Updating instance_info_cache with network_info: [{"id": "b58865c9-a096-40af-91cf-828e5131ea22", "address": "fa:16:3e:73:fe:b6", "network": {"id": "554fd203-5ff6-4282-87b1-d27aa5dba9e3", "bridge": "br-int", "label": "tempest-TestVolumeSwap-218878042-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.177", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f51faa215a1546fbaa0db25c9773ba6f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb58865c9-a0", "ovs_interfaceid": "b58865c9-a096-40af-91cf-828e5131ea22", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:32:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e6a62604-eda8-4c09-b1a4-20e7eaf2d06b req-9f8d6d9c-631c-4342-9242-9112bc711a9f service nova] Releasing lock "refresh_cache-ccf9f424-d230-4079-87c5-bb08bc85b0b7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:32:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-c5ec4ac4-48c4-45e6-bf3e-f3d8e28ac27a req-9878d4c3-ff51-45ce-afef-804e5e26966a service nova] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Received event network-changed-7b7ff2ca-c0a4-4414-9eed-62bd43be3613 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-c5ec4ac4-48c4-45e6-bf3e-f3d8e28ac27a req-9878d4c3-ff51-45ce-afef-804e5e26966a service nova] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Refreshing instance network info cache due to event network-changed-7b7ff2ca-c0a4-4414-9eed-62bd43be3613. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c5ec4ac4-48c4-45e6-bf3e-f3d8e28ac27a req-9878d4c3-ff51-45ce-afef-804e5e26966a service nova] Acquiring lock "refresh_cache-413850ca-8175-4ed6-a6cf-ff42230fa1e0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c5ec4ac4-48c4-45e6-bf3e-f3d8e28ac27a req-9878d4c3-ff51-45ce-afef-804e5e26966a service nova] Acquired lock "refresh_cache-413850ca-8175-4ed6-a6cf-ff42230fa1e0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG nova.network.neutron [req-c5ec4ac4-48c4-45e6-bf3e-f3d8e28ac27a req-9878d4c3-ff51-45ce-afef-804e5e26966a service nova] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Refreshing network info cache for port 7b7ff2ca-c0a4-4414-9eed-62bd43be3613 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wPNTeqDg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:32:28 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:32:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:32:28 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:32:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:32:28 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[369c8334-e2a6-44bc-b07a-bdab1dbff527]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:32:28 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2d5ecf64-b45c-4fef-9e3f-da919a7d12ef]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3] added
Apr 12 03:32:28 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5bb34999-d8ff-466b-bb13-2d725ee39599]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe8e1cd9-2934-49eb-8a56-8d2de5b3f59e]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v wPNTeqDg {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG nova.network.neutron [req-c5ec4ac4-48c4-45e6-bf3e-f3d8e28ac27a req-9878d4c3-ff51-45ce-afef-804e5e26966a service nova] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Updated VIF entry in instance network info cache for port 7b7ff2ca-c0a4-4414-9eed-62bd43be3613. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG nova.network.neutron [req-c5ec4ac4-48c4-45e6-bf3e-f3d8e28ac27a req-9878d4c3-ff51-45ce-afef-804e5e26966a service nova] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Updating instance_info_cache with network_info: [{"id": "7b7ff2ca-c0a4-4414-9eed-62bd43be3613", "address": "fa:16:3e:d2:cd:e1", "network": {"id": "01db77fb-7cb3-4eb1-b9d4-9ce9a671b333", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-539071762-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.170", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "90a92643aee44b66864aa10c2cf6f048", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7b7ff2ca-c0", "ovs_interfaceid": "7b7ff2ca-c0a4-4414-9eed-62bd43be3613", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v wPNTeqDg" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2a1afd49-ebe9-49ab-9f55-5e22ed6fa1e6]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'wPNTeqDg'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c5ec4ac4-48c4-45e6-bf3e-f3d8e28ac27a req-9878d4c3-ff51-45ce-afef-804e5e26966a service nova] Releasing lock "refresh_cache-413850ca-8175-4ed6-a6cf-ff42230fa1e0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b2dcfec-7f15-4f60-8113-5544efbf8e0f]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.033s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ba0f329e-601b-4b74-810e-ac90fa1ee127]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:28 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:28 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:28 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:28 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:28 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:28 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:28 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:28 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:28 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:28 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:28 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:28 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --login" returned: 0 in 0.092s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c865beae-80a0-455f-821f-a2e8d4cadb53]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3, portal: 172.16.0.220,3260]
Apr 12 03:32:28 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3, portal: 172.16.0.220,3260] successful.
Apr 12 03:32:28 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a8c44df-b38f-41c2-945a-ab413df8b008]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[eb442dd9-04bc-492b-8ea5-a648374c3e01]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:28 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:28 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:28 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:28 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:28 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:28 user nova-compute[72081]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:28 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:28 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:28 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:28 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:28 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:28 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:28 user nova-compute[72081]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:28 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.336s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] HCTL ('39', '-', '-', 0) found on session 9 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Scanning host 39 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host39/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host39/scan" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2d3045e9-a005-4cec-8b5c-6d01e6a7f4bb]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:28 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Searching for a device in session 9 and hctl ['39', '*', '*', 0] yield: None {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681270334.2632158, ef4ab4cb-6164-47ea-be1d-cbe0698f991d => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:32:29 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] VM Stopped (Lifecycle Event)
Apr 12 03:32:29 user nova-compute[72081]: DEBUG nova.compute.manager [None req-2bf8873b-9e65-4a05-9f4f-ecd218621480 None None] [instance: ef4ab4cb-6164-47ea-be1d-cbe0698f991d] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Acquiring lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lazy-loading 'flavor' on Instance uuid 413850ca-8175-4ed6-a6cf-ff42230fa1e0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.054s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Searching for a device in session 9 and hctl ['39', '*', '*', 0] yield: sdh {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Connected to sdh using {'target_iqn': 'iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wPNTeqDg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.366s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] <== connect_volume: return (1369ms) {'type': 'block', 'scsi_wwn': '23263376262653465', 'path': '/dev/sdh'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23263376262653465', 'path': '/dev/sdh'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG nova.objects.instance [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lazy-loading 'flavor' on Instance uuid ccf9f424-d230-4079-87c5-bb08bc85b0b7 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] attach device xml: <disk type="block" device="disk">
Apr 12 03:32:29 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:32:29 user nova-compute[72081]:   <source dev="/dev/sdh"/>
Apr 12 03:32:29 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:32:29 user nova-compute[72081]:   <serial>6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3</serial>
Apr 12 03:32:29 user nova-compute[72081]: </disk>
Apr 12 03:32:29 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Acquiring lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:29 user nova-compute[72081]: INFO nova.compute.manager [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Attaching volume 61ed3b81-1f3d-4068-aaa7-15aa2403a330 to /dev/sdc
Apr 12 03:32:29 user nova-compute[72081]: DEBUG os_brick.utils [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c596cd84-30d2-4b9e-a49f-ebea1200c0b0]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7edb495e-a868-4fb5-b50f-1a22f927d6f4]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[aef28cfc-89c3-45f4-9d30-b39291968efa]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[707a0b33-e77d-413c-b62f-48bc5f1e6a7b]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:32:29 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:32:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[00b9baa8-f461-49ac-b7df-1654d362b437]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG os_brick.utils [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] <== get_connector_properties: return (125ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:32:29 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Updating existing volume attachment record: 419ee285-2a87-4c3b-b456-c6f5bc9ae2e1 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:32:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:32:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:32:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] No VIF found with MAC fa:16:3e:73:fe:b6, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:32:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1bfc8386-b26b-4b0a-98ac-d2ce5f3ac4f2 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.486s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Acquiring lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7" by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7" acquired by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:32 user nova-compute[72081]: DEBUG os_brick.utils [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:32:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.012s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a1f70a09-f4e7-45bb-88a6-855c6db263b4]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3f98a56a-7c98-4350-a03a-7dbe142f13a1]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[91dac956-6dac-498d-983a-f4582fc42b20]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2a85019d-368f-479f-b6a2-6b59ed3c5112]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:32:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:32:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:32:32 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:32:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[afcdabfc-e02e-4f2c-adc3-f042bc314070]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:32:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:32:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:32:32 user nova-compute[72081]: DEBUG os_brick.utils [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] <== get_connector_properties: return (124ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:32:32 user nova-compute[72081]: INFO nova.compute.manager [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Swapping volume 6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 for 9a82b8b7-d64b-4407-869f-d5b1dcdf0550
Apr 12 03:32:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:32:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pLnUfzTt', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:32:36 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:32:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:32:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.003s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:32:36 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:32:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:32:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:32:36 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:32:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:36 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d5b0b770-99c0-4ad3-af28-0c799709c101]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:32:36 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:36 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e25bcdc3-2885-4bd7-88c7-f92c9db4eb56]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330] added
Apr 12 03:32:36 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:36 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[98b6f93f-3bdc-4876-9f9a-9f650c118f4b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:36 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b490cd7-8e62-4685-af99-90672449100b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v pLnUfzTt {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v pLnUfzTt" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:36 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3d61133e-b862-45cd-8f21-1aed6ae35e28]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'pLnUfzTt'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[92616ae7-b7de-46ac-89af-d6838a83ad19]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.031s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5e5423c-db47-4bca-a914-b174530a7311]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:37 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:37 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 -p 172.16.0.220:3260 --login" returned: 0 in 0.090s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8d41ad91-520d-4a2e-b362-ca0886ba5bd2]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330, portal: 172.16.0.220,3260]
Apr 12 03:32:37 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330, portal: 172.16.0.220,3260] successful.
Apr 12 03:32:37 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[897effad-cfc1-4ccb-80fd-b29f0640db26]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e7e2aa0-4890-46d8-b3a9-37b5bf06a1a5]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:37 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:37 user nova-compute[72081]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:37 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.336s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] HCTL ('40', '-', '-', 0) found on session 10 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Scanning host 40 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host40/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host40/scan" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[66546504-259a-4b91-8a7b-7668e9aaaa48]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Searching for a device in session 10 and hctl ['40', '*', '*', 0] yield: sdi {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Connected to sdi using {'target_iqn': 'iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pLnUfzTt', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:32:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] swap_volume: Calling driver volume swap with connection infos: new: {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kdDrZQOh', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'ccf9f424-d230-4079-87c5-bb08bc85b0b7', 'attached_at': '', 'detached_at': '', 'volume_id': '9a82b8b7-d64b-4407-869f-d5b1dcdf0550', 'serial': '9a82b8b7-d64b-4407-869f-d5b1dcdf0550'}; old: {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wPNTeqDg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdh'}, 'status': 'reserved', 'instance': 'ccf9f424-d230-4079-87c5-bb08bc85b0b7', 'attached_at': '', 'detached_at': '', 'volume_id': '6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3', 'serial'} {{(pid=72081) _swap_volume /opt/stack/nova/nova/compute/manager.py:7656}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kdDrZQOh', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:32:38 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:32:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.374s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] <== connect_volume: return (1377ms) {'type': 'block', 'scsi_wwn': '26564353532353134', 'path': '/dev/sdi'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26564353532353134', 'path': '/dev/sdi'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.005s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:32:38 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:32:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:32:38 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lazy-loading 'flavor' on Instance uuid 413850ca-8175-4ed6-a6cf-ff42230fa1e0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[87b6fd6d-09aa-4377-b3d3-3178390c82e4]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:32:38 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4422b9c2-44d4-45c3-a747-f8b19773c9db]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550] added
Apr 12 03:32:38 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] attach device xml: <disk type="block" device="disk">
Apr 12 03:32:38 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:32:38 user nova-compute[72081]:   <source dev="/dev/sdi"/>
Apr 12 03:32:38 user nova-compute[72081]:   <target dev="sdc" bus="scsi"/>
Apr 12 03:32:38 user nova-compute[72081]:   <serial>61ed3b81-1f3d-4068-aaa7-15aa2403a330</serial>
Apr 12 03:32:38 user nova-compute[72081]:   <address type="drive" controller="0" unit="2"/>
Apr 12 03:32:38 user nova-compute[72081]: </disk>
Apr 12 03:32:38 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[897f7815-64b9-4bdd-91b2-15e239bc037d]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f1aa7ca5-c120-4f1e-b605-2bba7f653b9a]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v kdDrZQOh {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v kdDrZQOh" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[dd693e6b-5065-4775-9aef-8428794b2d54]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'kdDrZQOh'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[eb40b48b-8b67-43b8-a69a-b439f500e80f]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0ca018b1-795c-4606-9eec-dd31bb1bc2f2]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 -p 172.16.0.220:3260 --login" returned: 0 in 0.084s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d3003dbf-e875-44b4-a814-6ab295626471]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550, portal: 172.16.0.220,3260]
Apr 12 03:32:38 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550, portal: 172.16.0.220,3260] successful.
Apr 12 03:32:38 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[29e96139-7814-4833-9be4-75ee606ee56c]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.031s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[07d1c30c-0d3a-4aed-ae05-1ba7f71b8d4e]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 (non-flash)\ntcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:38 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.319s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] HCTL ('41', '-', '-', 0) found on session 11 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Scanning host 41 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host41/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host41/scan" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ef503b5a-9bce-45ac-aa7c-d79a3d97212f]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Searching for a device in session 11 and hctl ['41', '*', '*', 0] yield: None {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] No BDM found with device name sda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] No BDM found with device name sdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] No BDM found with device name sdc, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] No VIF found with MAC fa:16:3e:d2:cd:e1, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:32:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4a8cc635-2a1f-4102-bff7-5b2c769ad5e3 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 9.137s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:39 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Searching for a device in session 11 and hctl ['41', '*', '*', 0] yield: sdj {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:32:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Connected to sdj using {'target_iqn': 'iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kdDrZQOh', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:32:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.348s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:32:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] <== connect_volume: return (1353ms) {'type': 'block', 'scsi_wwn': '23930353439613165', 'path': '/dev/sdj'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:32:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23930353439613165', 'path': '/dev/sdj'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:32:39 user nova-compute[72081]: DEBUG nova.objects.instance [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Lazy-loading 'flavor' on Instance uuid ccf9f424-d230-4079-87c5-bb08bc85b0b7 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:32:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Acquiring lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:39 user nova-compute[72081]: INFO nova.compute.manager [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Detaching volume 61ed3b81-1f3d-4068-aaa7-15aa2403a330
Apr 12 03:32:39 user nova-compute[72081]: INFO nova.virt.block_device [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Attempting to driver detach volume 61ed3b81-1f3d-4068-aaa7-15aa2403a330 from mountpoint /dev/sdc
Apr 12 03:32:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Attempting to detach device sdc from instance 413850ca-8175-4ed6-a6cf-ff42230fa1e0 from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 03:32:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:32:39 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:32:39 user nova-compute[72081]:   <source dev="/dev/sdi"/>
Apr 12 03:32:39 user nova-compute[72081]:   <target dev="sdc" bus="scsi"/>
Apr 12 03:32:39 user nova-compute[72081]:   <serial>61ed3b81-1f3d-4068-aaa7-15aa2403a330</serial>
Apr 12 03:32:39 user nova-compute[72081]:   <address type="drive" controller="0" bus="0" target="0" unit="2"/>
Apr 12 03:32:39 user nova-compute[72081]: </disk>
Apr 12 03:32:39 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:32:39 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Successfully detached device sdc from instance 413850ca-8175-4ed6-a6cf-ff42230fa1e0 from the persistent domain config.
Apr 12 03:32:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] (1/8): Attempting to detach device sdc with device alias scsi0-0-0-2 from instance 413850ca-8175-4ed6-a6cf-ff42230fa1e0 from the live domain config. {{(pid=72081) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 12 03:32:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:32:39 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:32:39 user nova-compute[72081]:   <source dev="/dev/sdi"/>
Apr 12 03:32:39 user nova-compute[72081]:   <target dev="sdc" bus="scsi"/>
Apr 12 03:32:39 user nova-compute[72081]:   <serial>61ed3b81-1f3d-4068-aaa7-15aa2403a330</serial>
Apr 12 03:32:39 user nova-compute[72081]:   <address type="drive" controller="0" bus="0" target="0" unit="2"/>
Apr 12 03:32:39 user nova-compute[72081]: </disk>
Apr 12 03:32:39 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] COPY block job progress, current cursor: 6291456 final cursor: 1073741824 {{(pid=72081) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Received event <DeviceRemovedEvent: 1681270360.3498678, 413850ca-8175-4ed6-a6cf-ff42230fa1e0 => scsi0-0-0-2> from libvirt while the driver is waiting for it; dispatched. {{(pid=72081) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Start waiting for the detach event from libvirt for device sdc with device alias scsi0-0-0-2 for instance 413850ca-8175-4ed6-a6cf-ff42230fa1e0 {{(pid=72081) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 12 03:32:40 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Successfully detached device sdc from instance 413850ca-8175-4ed6-a6cf-ff42230fa1e0 from the live domain config.
Apr 12 03:32:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pLnUfzTt', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdi': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:32:40 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:32:40 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.030s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[580c9427-fbcd-4834-a88e-522d95d3d4a7]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:32:40 user nova-compute[72081]: No targets found.
Apr 12 03:32:40 user nova-compute[72081]: iSNS:
Apr 12 03:32:40 user nova-compute[72081]: No targets found.
Apr 12 03:32:40 user nova-compute[72081]: STATIC:
Apr 12 03:32:40 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550
Apr 12 03:32:40 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:40 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:40 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9
Apr 12 03:32:40 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:40 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:40 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1
Apr 12 03:32:40 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:40 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:40 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3
Apr 12 03:32:40 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:40 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:40 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3
Apr 12 03:32:40 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:40 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:40 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d
Apr 12 03:32:40 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:40 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:40 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd
Apr 12 03:32:40 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:40 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:40 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352
Apr 12 03:32:40 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:40 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:40 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330
Apr 12 03:32:40 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:40 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:40 user nova-compute[72081]: FIRMWARE:
Apr 12 03:32:40 user nova-compute[72081]: No targets found.
Apr 12 03:32:40 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:32:40 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:32:40 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c84c1a28-a18e-4bd5-b9f7-37c36c54644d]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.033s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ba8c8296-3a0f-4509-b83e-948512582f1d]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 (non-flash)\ntcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 (non-flash)
Apr 12 03:32:40 user nova-compute[72081]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)
Apr 12 03:32:40 user nova-compute[72081]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:40 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:40 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:40 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:40 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:40 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:40 user nova-compute[72081]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:40 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 (non-flash)
Apr 12 03:32:40 user nova-compute[72081]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)
Apr 12 03:32:40 user nova-compute[72081]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:40 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:40 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:40 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:40 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:40 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:40 user nova-compute[72081]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:40 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fac2ccdc0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330'): ({'sdi'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Removing single pathed devices sdi {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[53497381-2eea-4268-aceb-ae6e03f547d4]: (4, ('path checker states:\nup                  9\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdi" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bc2dad88-1e99-49f7-bf50-fddb5cc9e217]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Flushing IO for device /dev/sdi {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] COPY block job progress, current cursor: 156237824 final cursor: 1073741824 {{(pid=72081) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdi" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2ee3f7b1-a4fb-43d5-a245-8326a739ac79]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Remove SCSI device /dev/sdi with /sys/block/sdi/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdi/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdi/device/delete" returned: 0 in 0.043s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[05554991-93ff-4748-a90a-801510065fdf]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Checking to see if SCSI volumes sdi have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] SCSI volumes sdi have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.032s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9a72fc6d-e3da-4e6e-87da-23f33e6b0bd9]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 -p 172.16.0.220:3260 --logout" returned: 0 in 0.095s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[80b1c901-b531-4e81-8074-552fe7c9d320]: (4, ('Logging out of session [sid: 10, target: iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330, portal: 172.16.0.220,3260]\nLogout of [sid: 10, target: iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 10, target: iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330, portal: 172.16.0.220,3260]
Apr 12 03:32:40 user nova-compute[72081]: Logout of [sid: 10, target: iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330, portal: 172.16.0.220,3260] successful.
Apr 12 03:32:40 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:61ed3b81-1f3d-4068-aaa7-15aa2403a330 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.048s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[cf2c6a01-55c7-4f3e-af9f-136144e370d5]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.466s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] <== disconnect_volume: return (467ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:32:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:32:41 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] COPY block job progress, current cursor: 306184192 final cursor: 1073741824 {{(pid=72081) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 12 03:32:41 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] COPY block job progress, current cursor: 469762048 final cursor: 1073741824 {{(pid=72081) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 12 03:32:41 user nova-compute[72081]: DEBUG nova.objects.instance [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lazy-loading 'flavor' on Instance uuid 413850ca-8175-4ed6-a6cf-ff42230fa1e0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:32:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f6c3065b-988d-40c3-9a56-dbd36b4b0912 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 2.104s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] COPY block job progress, current cursor: 615514112 final cursor: 1073741824 {{(pid=72081) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 12 03:32:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4593571d-cd4e-4242-8aeb-e6c5424f0bb7 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Acquiring lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4593571d-cd4e-4242-8aeb-e6c5424f0bb7 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.003s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4593571d-cd4e-4242-8aeb-e6c5424f0bb7 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Acquiring lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4593571d-cd4e-4242-8aeb-e6c5424f0bb7 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4593571d-cd4e-4242-8aeb-e6c5424f0bb7 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:42 user nova-compute[72081]: INFO nova.compute.manager [None req-4593571d-cd4e-4242-8aeb-e6c5424f0bb7 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Terminating instance
Apr 12 03:32:42 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4593571d-cd4e-4242-8aeb-e6c5424f0bb7 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:32:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] COPY block job progress, current cursor: 816840704 final cursor: 1073741824 {{(pid=72081) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 12 03:32:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] COPY block job progress, current cursor: 961544192 final cursor: 1073741824 {{(pid=72081) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG nova.compute.manager [req-15e5a08d-a7ba-4833-b593-7eec3a0b953d req-924343be-0f1c-4b16-9683-ec60c7475b39 service nova] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Received event network-vif-unplugged-7b7ff2ca-c0a4-4414-9eed-62bd43be3613 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-15e5a08d-a7ba-4833-b593-7eec3a0b953d req-924343be-0f1c-4b16-9683-ec60c7475b39 service nova] Acquiring lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-15e5a08d-a7ba-4833-b593-7eec3a0b953d req-924343be-0f1c-4b16-9683-ec60c7475b39 service nova] Lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-15e5a08d-a7ba-4833-b593-7eec3a0b953d req-924343be-0f1c-4b16-9683-ec60c7475b39 service nova] Lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG nova.compute.manager [req-15e5a08d-a7ba-4833-b593-7eec3a0b953d req-924343be-0f1c-4b16-9683-ec60c7475b39 service nova] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] No waiting events found dispatching network-vif-unplugged-7b7ff2ca-c0a4-4414-9eed-62bd43be3613 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG nova.compute.manager [req-15e5a08d-a7ba-4833-b593-7eec3a0b953d req-924343be-0f1c-4b16-9683-ec60c7475b39 service nova] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Received event network-vif-unplugged-7b7ff2ca-c0a4-4414-9eed-62bd43be3613 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:32:43 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Instance destroyed successfully.
Apr 12 03:32:43 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4593571d-cd4e-4242-8aeb-e6c5424f0bb7 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lazy-loading 'resources' on Instance uuid 413850ca-8175-4ed6-a6cf-ff42230fa1e0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-4593571d-cd4e-4242-8aeb-e6c5424f0bb7 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:30:36Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachSCSIVolumeTestJSON-server-1054497631',display_name='tempest-AttachSCSIVolumeTestJSON-server-1054497631',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachscsivolumetestjson-server-1054497631',id=7,image_ref='6aba2abf-651e-4da7-9a27-c2c991cb5c61',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEW7Yrpu5SOQ/b67vmyb/sqyksJOlf4MPK63MIfzlSnm+OH9BJ8Yy9IStENYg+9gKakI2VPnKTMAt5l9nxKRjD2o/A0rxASwuMLoI7j1zUAFxuHzukBR9RVxm++z5vy2Ww==',key_name='tempest-keypair-539207312',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:30:48Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='90a92643aee44b66864aa10c2cf6f048',ramdisk_id='',reservation_id='r-dosfn8tv',resources=None,root_device_name='/dev/sda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='6aba2abf-651e-4da7-9a27-c2c991cb5c61',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='scsi',image_hw_disk_bus='scsi',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_scsi_model='virtio-scsi',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-AttachSCSIVolumeTestJSON-95582294',owner_user_name='tempest-AttachSCSIVolumeTestJSON-95582294-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:30:48Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='431c372c4da24fa692f43f2d16764755',uuid=413850ca-8175-4ed6-a6cf-ff42230fa1e0,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "7b7ff2ca-c0a4-4414-9eed-62bd43be3613", "address": "fa:16:3e:d2:cd:e1", "network": {"id": "01db77fb-7cb3-4eb1-b9d4-9ce9a671b333", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-539071762-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.170", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "90a92643aee44b66864aa10c2cf6f048", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7b7ff2ca-c0", "ovs_interfaceid": "7b7ff2ca-c0a4-4414-9eed-62bd43be3613", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4593571d-cd4e-4242-8aeb-e6c5424f0bb7 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Converting VIF {"id": "7b7ff2ca-c0a4-4414-9eed-62bd43be3613", "address": "fa:16:3e:d2:cd:e1", "network": {"id": "01db77fb-7cb3-4eb1-b9d4-9ce9a671b333", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-539071762-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.170", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "90a92643aee44b66864aa10c2cf6f048", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7b7ff2ca-c0", "ovs_interfaceid": "7b7ff2ca-c0a4-4414-9eed-62bd43be3613", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4593571d-cd4e-4242-8aeb-e6c5424f0bb7 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:d2:cd:e1,bridge_name='br-int',has_traffic_filtering=True,id=7b7ff2ca-c0a4-4414-9eed-62bd43be3613,network=Network(01db77fb-7cb3-4eb1-b9d4-9ce9a671b333),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7b7ff2ca-c0') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG os_vif [None req-4593571d-cd4e-4242-8aeb-e6c5424f0bb7 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:d2:cd:e1,bridge_name='br-int',has_traffic_filtering=True,id=7b7ff2ca-c0a4-4414-9eed-62bd43be3613,network=Network(01db77fb-7cb3-4eb1-b9d4-9ce9a671b333),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7b7ff2ca-c0') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap7b7ff2ca-c0, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:43 user nova-compute[72081]: INFO os_vif [None req-4593571d-cd4e-4242-8aeb-e6c5424f0bb7 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:d2:cd:e1,bridge_name='br-int',has_traffic_filtering=True,id=7b7ff2ca-c0a4-4414-9eed-62bd43be3613,network=Network(01db77fb-7cb3-4eb1-b9d4-9ce9a671b333),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7b7ff2ca-c0')
Apr 12 03:32:43 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-4593571d-cd4e-4242-8aeb-e6c5424f0bb7 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Deleting instance files /opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0_del
Apr 12 03:32:43 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-4593571d-cd4e-4242-8aeb-e6c5424f0bb7 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Deletion of /opt/stack/data/nova/instances/413850ca-8175-4ed6-a6cf-ff42230fa1e0_del complete
Apr 12 03:32:43 user nova-compute[72081]: INFO nova.compute.manager [None req-4593571d-cd4e-4242-8aeb-e6c5424f0bb7 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Took 0.91 seconds to destroy the instance on the hypervisor.
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-4593571d-cd4e-4242-8aeb-e6c5424f0bb7 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] COPY block job progress, current cursor: 1073741824 final cursor: 1073741824 {{(pid=72081) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wPNTeqDg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdh': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:32:43 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:32:43 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[22d42f65-87d5-4b83-83f7-48b5d6e6dcf9]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:32:43 user nova-compute[72081]: No targets found.
Apr 12 03:32:43 user nova-compute[72081]: iSNS:
Apr 12 03:32:43 user nova-compute[72081]: No targets found.
Apr 12 03:32:43 user nova-compute[72081]: STATIC:
Apr 12 03:32:43 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550
Apr 12 03:32:43 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:43 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:43 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9
Apr 12 03:32:43 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:43 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:43 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1
Apr 12 03:32:43 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:43 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:43 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3
Apr 12 03:32:43 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:43 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:43 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3
Apr 12 03:32:43 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:43 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:43 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d
Apr 12 03:32:43 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:43 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:43 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd
Apr 12 03:32:43 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:43 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:43 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352
Apr 12 03:32:43 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:43 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:43 user nova-compute[72081]: FIRMWARE:
Apr 12 03:32:43 user nova-compute[72081]: No targets found.
Apr 12 03:32:43 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:32:43 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:32:43 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a51e46dd-d369-46af-93b9-431a73945283]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e619ae08-3df1-441b-9cd9-7cb9823a0a6f]: (4, ('tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)
Apr 12 03:32:43 user nova-compute[72081]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:43 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:43 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:43 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:43 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:43 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:43 user nova-compute[72081]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:43 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsi session list stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)
Apr 12 03:32:43 user nova-compute[72081]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:43 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:43 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:43 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:43 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:43 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:43 user nova-compute[72081]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:43 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fb41e1d80>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3'): ({'sdh'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Removing single pathed devices sdh {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[446cc027-7722-45b2-8f76-11a9c0883ecc]: (4, ('path checker states:\nup                  8\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdh {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdh" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d041eb93-0219-41d0-a655-82d1c2d3739b]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Flushing IO for device /dev/sdh {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdh {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdh" returned: 0 in 0.012s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[215d3ad6-05a5-4663-a845-44dc0696372a]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Remove SCSI device /dev/sdh with /sys/block/sdh/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdh/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdh/device/delete" returned: 0 in 0.033s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e70c447a-e21a-4e44-b3d6-d2982d0a1eba]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Checking to see if SCSI volumes sdh have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] SCSI volumes sdh have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[49602951-73ff-4c40-9600-3292c57af443]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:43 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:44 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --logout" returned: 0 in 0.073s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:44 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[37ae474d-9e67-460c-9239-a6c58e9defce]: (4, ('Logging out of session [sid: 9, target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3, portal: 172.16.0.220,3260]\nLogout of [sid: 9, target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:44 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('--logout',): stdout=Logging out of session [sid: 9, target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3, portal: 172.16.0.220,3260]
Apr 12 03:32:44 user nova-compute[72081]: Logout of [sid: 9, target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3, portal: 172.16.0.220,3260] successful.
Apr 12 03:32:44 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:44 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:44 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.034s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:44 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f515054d-eed3-471d-be64-19b14025ea53]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:44 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:44 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.318s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:32:44 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] <== disconnect_volume: return (319ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:32:44 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:32:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] swap_volume: Driver volume swap returned, new connection_info is now : {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kdDrZQOh', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdj'}, 'status': 'reserved', 'instance': 'ccf9f424-d230-4079-87c5-bb08bc85b0b7', 'attached_at': '', 'detached_at': '', 'volume_id': '9a82b8b7-d64b-4407-869f-d5b1dcdf0550', 'serial'} {{(pid=72081) _swap_volume /opt/stack/nova/nova/compute/manager.py:7665}}
Apr 12 03:32:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] swap_volume: removing Cinder connection for volume 6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 {{(pid=72081) _swap_volume /opt/stack/nova/nova/compute/manager.py:7710}}
Apr 12 03:32:44 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:32:44 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Took 1.20 seconds to deallocate network for instance.
Apr 12 03:32:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4593571d-cd4e-4242-8aeb-e6c5424f0bb7 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4593571d-cd4e-4242-8aeb-e6c5424f0bb7 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:44 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-4593571d-cd4e-4242-8aeb-e6c5424f0bb7 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:32:44 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-4593571d-cd4e-4242-8aeb-e6c5424f0bb7 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:32:45 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] swap_volume: Updating volume 6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 BDM record with {'connection_info': '{"driver_volume_type": "iscsi", "data": {"target_iqn": "iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550", "target_portal": "172.16.0.220:3260", "target_discovered": false, "auth_method": "CHAP", "auth_username": "kdDrZQOh", "auth_password": "2oY4VKwJ6XO1hM", "target_lun": 0, "qos_specs": null, "access_mode": "rw", "encrypted": false, "cacheable": false, "device_path": "/dev/sdj", "volume_id": "9a82b8b7-d64b-4407-869f-d5b1dcdf0550"}, "status": "reserved", "instance": "ccf9f424-d230-4079-87c5-bb08bc85b0b7", "attached_at": "", "detached_at": "", "volume_id": "9a82b8b7-d64b-4407-869f-d5b1dcdf0550", "serial": "9a82b8b7-d64b-4407-869f-d5b1dcdf0550"}', 'source_type': 'volume', 'destination_type': 'volume', 'snapshot_id': None, 'volume_id': '9a82b8b7-d64b-4407-869f-d5b1dcdf0550', 'no_device': None, 'attachment_id': 'dae870f8-b0d7-49cb-8b09-1e6c7630d2c2'} {{(pid=72081) _do_swap_volume /opt/stack/nova/nova/compute/manager.py:7858}}
Apr 12 03:32:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ee583708-0bae-4adf-8e68-15e43a101493 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7" "released" by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" :: held 12.981s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4593571d-cd4e-4242-8aeb-e6c5424f0bb7 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.308s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:45 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-4593571d-cd4e-4242-8aeb-e6c5424f0bb7 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Deleted allocations for instance 413850ca-8175-4ed6-a6cf-ff42230fa1e0
Apr 12 03:32:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4593571d-cd4e-4242-8aeb-e6c5424f0bb7 tempest-AttachSCSIVolumeTestJSON-95582294 tempest-AttachSCSIVolumeTestJSON-95582294-project-member] Lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.622s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:45 user nova-compute[72081]: DEBUG nova.compute.manager [req-a38460b2-b9e8-4efc-99f7-60cde1ad6dc7 req-42a72de7-2786-42be-83ca-87f200f014d8 service nova] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Received event network-vif-plugged-7b7ff2ca-c0a4-4414-9eed-62bd43be3613 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a38460b2-b9e8-4efc-99f7-60cde1ad6dc7 req-42a72de7-2786-42be-83ca-87f200f014d8 service nova] Acquiring lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a38460b2-b9e8-4efc-99f7-60cde1ad6dc7 req-42a72de7-2786-42be-83ca-87f200f014d8 service nova] Lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a38460b2-b9e8-4efc-99f7-60cde1ad6dc7 req-42a72de7-2786-42be-83ca-87f200f014d8 service nova] Lock "413850ca-8175-4ed6-a6cf-ff42230fa1e0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:45 user nova-compute[72081]: DEBUG nova.compute.manager [req-a38460b2-b9e8-4efc-99f7-60cde1ad6dc7 req-42a72de7-2786-42be-83ca-87f200f014d8 service nova] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] No waiting events found dispatching network-vif-plugged-7b7ff2ca-c0a4-4414-9eed-62bd43be3613 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:45 user nova-compute[72081]: WARNING nova.compute.manager [req-a38460b2-b9e8-4efc-99f7-60cde1ad6dc7 req-42a72de7-2786-42be-83ca-87f200f014d8 service nova] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Received unexpected event network-vif-plugged-7b7ff2ca-c0a4-4414-9eed-62bd43be3613 for instance with vm_state deleted and task_state None.
Apr 12 03:32:45 user nova-compute[72081]: DEBUG nova.compute.manager [req-a38460b2-b9e8-4efc-99f7-60cde1ad6dc7 req-42a72de7-2786-42be-83ca-87f200f014d8 service nova] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Received event network-vif-deleted-7b7ff2ca-c0a4-4414-9eed-62bd43be3613 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:46 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Acquiring lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7" by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:46 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7" acquired by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:46 user nova-compute[72081]: DEBUG os_brick.utils [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:32:46 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:46 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:46 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d6b1d1a4-6b3f-47d4-adff-25d49def31c9]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:46 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:46 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:46 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e2b883bf-56c2-4911-b3ec-c548bb2c1f05]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:46 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:46 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:46 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1b9f7aee-fed9-4b95-b2fe-4038a986e1df]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:46 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb028a43-8eb5-45ab-9322-40aa360cc0ce]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:46 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:46 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:32:46 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:32:46 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:46 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:32:46 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:32:46 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d9a1aad4-46a4-4c27-940e-192adf0d212c]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:46 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:32:46 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:32:46 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:32:46 user nova-compute[72081]: DEBUG os_brick.utils [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] <== get_connector_properties: return (111ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:32:46 user nova-compute[72081]: INFO nova.compute.manager [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Swapping volume 9a82b8b7-d64b-4407-869f-d5b1dcdf0550 for 6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3
Apr 12 03:32:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:47 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lazy-loading 'flavor' on Instance uuid cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:32:47 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Ignoring supplied device name: /dev/vdb
Apr 12 03:32:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.056s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:47 user nova-compute[72081]: INFO nova.compute.manager [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Attaching volume 263426d6-d387-4a67-9013-5a23e8386c93 to /dev/vdb
Apr 12 03:32:47 user nova-compute[72081]: DEBUG os_brick.utils [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:32:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:47 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4640a9cc-9b51-496c-8a19-77b41394ff05]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:47 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9e144bfe-1155-4a27-a768-1726e83f5969]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:47 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1404d8c8-cc15-4073-a62d-c47b475516f1]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:47 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[adfbfeaf-9dd0-42c3-9070-7b3d0cc7c136]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:32:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:32:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:32:47 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:32:47 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6402d857-e3d4-44e7-84ee-501421c6e681]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:32:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:32:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:32:47 user nova-compute[72081]: DEBUG os_brick.utils [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] <== get_connector_properties: return (115ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:32:47 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Updating existing volume attachment record: 96baaff2-cf6c-4bb9-bf8b-f22cd9f31adc {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:32:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:48 user nova-compute[72081]: DEBUG nova.compute.manager [req-b773a5d3-7b8c-4704-a6b4-95c5a01fd1fb req-a90f1dae-4504-4e5c-92e1-cf35bc1e2e5c service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-changed-ce9193f1-28a9-4c0f-95ae-e21f8140de19 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:48 user nova-compute[72081]: DEBUG nova.compute.manager [req-b773a5d3-7b8c-4704-a6b4-95c5a01fd1fb req-a90f1dae-4504-4e5c-92e1-cf35bc1e2e5c service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Refreshing instance network info cache due to event network-changed-ce9193f1-28a9-4c0f-95ae-e21f8140de19. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:32:48 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b773a5d3-7b8c-4704-a6b4-95c5a01fd1fb req-a90f1dae-4504-4e5c-92e1-cf35bc1e2e5c service nova] Acquiring lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:32:48 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b773a5d3-7b8c-4704-a6b4-95c5a01fd1fb req-a90f1dae-4504-4e5c-92e1-cf35bc1e2e5c service nova] Acquired lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:32:48 user nova-compute[72081]: DEBUG nova.network.neutron [req-b773a5d3-7b8c-4704-a6b4-95c5a01fd1fb req-a90f1dae-4504-4e5c-92e1-cf35bc1e2e5c service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Refreshing network info cache for port ce9193f1-28a9-4c0f-95ae-e21f8140de19 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:32:49 user nova-compute[72081]: DEBUG nova.network.neutron [req-b773a5d3-7b8c-4704-a6b4-95c5a01fd1fb req-a90f1dae-4504-4e5c-92e1-cf35bc1e2e5c service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Updated VIF entry in instance network info cache for port ce9193f1-28a9-4c0f-95ae-e21f8140de19. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:32:49 user nova-compute[72081]: DEBUG nova.network.neutron [req-b773a5d3-7b8c-4704-a6b4-95c5a01fd1fb req-a90f1dae-4504-4e5c-92e1-cf35bc1e2e5c service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Updating instance_info_cache with network_info: [{"id": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "address": "fa:16:3e:30:84:b9", "network": {"id": "9ea0ab38-8496-4a4b-a54b-fa36e90fd60a", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-849127753-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.24", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapce9193f1-28", "ovs_interfaceid": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "address": "fa:16:3e:56:d1:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.118", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap835b5a2a-4d", "ovs_interfaceid": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "a1aaff41-e246-4887-89c6-a91163d4efd5", "address": "fa:16:3e:d2:e8:92", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.42", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa1aaff41-e2", "ovs_interfaceid": "a1aaff41-e246-4887-89c6-a91163d4efd5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "ac401bfa-a662-4290-af6e-0a39e8fde61c", "address": "fa:16:3e:db:3e:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.161", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapac401bfa-a6", "ovs_interfaceid": "ac401bfa-a662-4290-af6e-0a39e8fde61c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "4e57de22-e876-4bfe-bd07-a1003afd2295", "address": "fa:16:3e:93:42:94", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4e57de22-e8", "ovs_interfaceid": "4e57de22-e876-4bfe-bd07-a1003afd2295", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8248913f-46c8-4f3d-ab47-0203cb267169", "address": "fa:16:3e:21:f0:77", "network": {"id": "435f9638-5e8f-40a8-b271-a183982c70d5", "bridge": "br-int", "label": "tempest-device-tagging-net2-1738895025", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8248913f-46", "ovs_interfaceid": "8248913f-46c8-4f3d-ab47-0203cb267169", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "21860860-2a08-43dc-bda8-0a1d0fab0634", "address": "fa:16:3e:eb:f1:cc", "network": {"id": "435f9638-5e8f-40a8-b271-a183982c70d5", "bridge": "br-int", "label": "tempest-device-tagging-net2-1738895025", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap21860860-2a", "ovs_interfaceid": "21860860-2a08-43dc-bda8-0a1d0fab0634", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:32:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b773a5d3-7b8c-4704-a6b4-95c5a01fd1fb req-a90f1dae-4504-4e5c-92e1-cf35bc1e2e5c service nova] Releasing lock "refresh_cache-7119b8b3-291b-4868-820e-d7bd3dd3ada4" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:32:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:50 user nova-compute[72081]: INFO nova.compute.manager [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Terminating instance
Apr 12 03:32:50 user nova-compute[72081]: DEBUG nova.compute.manager [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:32:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:50 user nova-compute[72081]: DEBUG nova.compute.manager [req-6b5755f1-551c-4d79-8fb2-6175b30952b4 req-adfb8c9c-ac7c-4f01-9c13-bcb1c56e975d service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-unplugged-ce9193f1-28a9-4c0f-95ae-e21f8140de19 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6b5755f1-551c-4d79-8fb2-6175b30952b4 req-adfb8c9c-ac7c-4f01-9c13-bcb1c56e975d service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6b5755f1-551c-4d79-8fb2-6175b30952b4 req-adfb8c9c-ac7c-4f01-9c13-bcb1c56e975d service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6b5755f1-551c-4d79-8fb2-6175b30952b4 req-adfb8c9c-ac7c-4f01-9c13-bcb1c56e975d service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:50 user nova-compute[72081]: DEBUG nova.compute.manager [req-6b5755f1-551c-4d79-8fb2-6175b30952b4 req-adfb8c9c-ac7c-4f01-9c13-bcb1c56e975d service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-unplugged-ce9193f1-28a9-4c0f-95ae-e21f8140de19 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:50 user nova-compute[72081]: DEBUG nova.compute.manager [req-6b5755f1-551c-4d79-8fb2-6175b30952b4 req-adfb8c9c-ac7c-4f01-9c13-bcb1c56e975d service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-unplugged-ce9193f1-28a9-4c0f-95ae-e21f8140de19 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:32:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG nova.compute.manager [req-75b4cafc-acd3-4815-aff7-75bbc4e8463a req-1b76245b-c9e7-4633-981a-491c34b13693 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-unplugged-a1aaff41-e246-4887-89c6-a91163d4efd5 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-75b4cafc-acd3-4815-aff7-75bbc4e8463a req-1b76245b-c9e7-4633-981a-491c34b13693 service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-75b4cafc-acd3-4815-aff7-75bbc4e8463a req-1b76245b-c9e7-4633-981a-491c34b13693 service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.004s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-75b4cafc-acd3-4815-aff7-75bbc4e8463a req-1b76245b-c9e7-4633-981a-491c34b13693 service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG nova.compute.manager [req-75b4cafc-acd3-4815-aff7-75bbc4e8463a req-1b76245b-c9e7-4633-981a-491c34b13693 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-unplugged-a1aaff41-e246-4887-89c6-a91163d4efd5 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG nova.compute.manager [req-75b4cafc-acd3-4815-aff7-75bbc4e8463a req-1b76245b-c9e7-4633-981a-491c34b13693 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-unplugged-a1aaff41-e246-4887-89c6-a91163d4efd5 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG nova.compute.manager [req-1a0a751b-e855-49e4-b63c-89f2f7ecbe91 req-5936891b-6072-448a-a064-c8cd2d04c333 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-unplugged-ac401bfa-a662-4290-af6e-0a39e8fde61c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a0a751b-e855-49e4-b63c-89f2f7ecbe91 req-5936891b-6072-448a-a064-c8cd2d04c333 service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a0a751b-e855-49e4-b63c-89f2f7ecbe91 req-5936891b-6072-448a-a064-c8cd2d04c333 service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a0a751b-e855-49e4-b63c-89f2f7ecbe91 req-5936891b-6072-448a-a064-c8cd2d04c333 service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG nova.compute.manager [req-1a0a751b-e855-49e4-b63c-89f2f7ecbe91 req-5936891b-6072-448a-a064-c8cd2d04c333 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-unplugged-ac401bfa-a662-4290-af6e-0a39e8fde61c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG nova.compute.manager [req-1a0a751b-e855-49e4-b63c-89f2f7ecbe91 req-5936891b-6072-448a-a064-c8cd2d04c333 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-unplugged-ac401bfa-a662-4290-af6e-0a39e8fde61c for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] swap_volume: Calling driver volume swap with connection infos: new: {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wPNTeqDg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'ccf9f424-d230-4079-87c5-bb08bc85b0b7', 'attached_at': '', 'detached_at': '', 'volume_id': '6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3', 'serial': '6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3'}; old: {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kdDrZQOh', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdj', 'volume_id': '9a82b8b7-d64b-4407-869f-d5b1dcdf0550'}, 'status': 'reserved', 'instance': 'ccf9f424-d230-4079-87c5-bb08bc85b0b7', 'attached_at': '', 'detached_at': '', 'volume_id': '9a82b8b7-d64b-4407-869f-d5b1dcdf0550', 'serial'} {{(pid=72081) _swap_volume /opt/stack/nova/nova/compute/manager.py:7656}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wPNTeqDg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:32:51 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:32:51 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.003s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:32:51 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:32:51 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:32:51 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260" returned: 21 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[db87cbc7-33c6-4f1e-bb90-ceaf6317c451]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:32:51 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b95cbd62-8b36-4b24-bdcd-8b119a4f7210]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3] added
Apr 12 03:32:51 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c96e0050-ddca-448b-8c89-2186a5e87533]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[95ca035c-8cfb-422e-9812-a327c632d2ca]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v wPNTeqDg {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v wPNTeqDg" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[59553406-b07a-4f11-91ea-32df00c386d6]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'wPNTeqDg'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e5df8643-6180-400f-8f0f-fa10252e534d]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.033s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[11562dfb-3073-401e-92eb-fda82336a70b]: (4, ('tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:51 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsi session list stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:51 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --login" returned: 0 in 0.105s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[144d2f5f-2e1e-443f-886f-82d5fd8ea310]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3, portal: 172.16.0.220,3260]
Apr 12 03:32:51 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3, portal: 172.16.0.220,3260] successful.
Apr 12 03:32:51 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f5b9ab1-6378-4e9d-8295-0d76731def06]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.035s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[63563652-6d92-4833-9775-052def5ce5c8]: (4, ('tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)\ntcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:51 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsi session list stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:51 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:51 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.387s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] HCTL ('39', '-', '-', 0) found on session 12 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Scanning host 39 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host39/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host39/scan" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[66cf0cbb-1fba-4ea9-b993-a3dde4684397]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Searching for a device in session 12 and hctl ['39', '*', '*', 0] yield: sdh {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:32:51 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Connected to sdh using {'target_iqn': 'iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wPNTeqDg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:32:52 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Instance destroyed successfully.
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.objects.instance [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lazy-loading 'resources' on Instance uuid 7119b8b3-291b-4868-820e-d7bd3dd3ada4 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:30:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1028113698',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1028113698',id=9,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS3ZbVHfxQVFHpZS5QTkoqKDiEjmhCdyP6+fgfZz0jwXQ4fAjcP60l9wC1gyScpF8Zw9rmHw2YJpeRpTUHuz2Hroka/B8qwVo0YxuHbLGdfKbP5YGwIEhUARrat7zqI4A==',key_name='tempest-keypair-476892975',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:31:36Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='72d31bafe182459aa8bcb4f1bbb5e6d0',ramdisk_id='',reservation_id='r-b6ok04uz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-294347777',owner_user_name='tempest-TaggedBootDevicesTest_v242-294347777-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:31:36Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e7aeea82653d4681aa22f6dd8ae742b4',uuid=7119b8b3-291b-4868-820e-d7bd3dd3ada4,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "address": "fa:16:3e:30:84:b9", "network": {"id": "9ea0ab38-8496-4a4b-a54b-fa36e90fd60a", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-849127753-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.24", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapce9193f1-28", "ovs_interfaceid": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converting VIF {"id": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "address": "fa:16:3e:30:84:b9", "network": {"id": "9ea0ab38-8496-4a4b-a54b-fa36e90fd60a", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-849127753-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.24", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapce9193f1-28", "ovs_interfaceid": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:30:84:b9,bridge_name='br-int',has_traffic_filtering=True,id=ce9193f1-28a9-4c0f-95ae-e21f8140de19,network=Network(9ea0ab38-8496-4a4b-a54b-fa36e90fd60a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapce9193f1-28') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG os_vif [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:30:84:b9,bridge_name='br-int',has_traffic_filtering=True,id=ce9193f1-28a9-4c0f-95ae-e21f8140de19,network=Network(9ea0ab38-8496-4a4b-a54b-fa36e90fd60a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapce9193f1-28') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapce9193f1-28, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:52 user nova-compute[72081]: INFO os_vif [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:30:84:b9,bridge_name='br-int',has_traffic_filtering=True,id=ce9193f1-28a9-4c0f-95ae-e21f8140de19,network=Network(9ea0ab38-8496-4a4b-a54b-fa36e90fd60a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapce9193f1-28')
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:30:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1028113698',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1028113698',id=9,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS3ZbVHfxQVFHpZS5QTkoqKDiEjmhCdyP6+fgfZz0jwXQ4fAjcP60l9wC1gyScpF8Zw9rmHw2YJpeRpTUHuz2Hroka/B8qwVo0YxuHbLGdfKbP5YGwIEhUARrat7zqI4A==',key_name='tempest-keypair-476892975',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:31:36Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='72d31bafe182459aa8bcb4f1bbb5e6d0',ramdisk_id='',reservation_id='r-b6ok04uz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-294347777',owner_user_name='tempest-TaggedBootDevicesTest_v242-294347777-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:31:36Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e7aeea82653d4681aa22f6dd8ae742b4',uuid=7119b8b3-291b-4868-820e-d7bd3dd3ada4,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "address": "fa:16:3e:56:d1:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.118", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap835b5a2a-4d", "ovs_interfaceid": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converting VIF {"id": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "address": "fa:16:3e:56:d1:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.118", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap835b5a2a-4d", "ovs_interfaceid": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:56:d1:90,bridge_name='br-int',has_traffic_filtering=True,id=835b5a2a-4d35-4d2a-9fdd-0c78b7def163,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap835b5a2a-4d') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG os_vif [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:56:d1:90,bridge_name='br-int',has_traffic_filtering=True,id=835b5a2a-4d35-4d2a-9fdd-0c78b7def163,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap835b5a2a-4d') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap835b5a2a-4d, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:52 user nova-compute[72081]: INFO os_vif [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:56:d1:90,bridge_name='br-int',has_traffic_filtering=True,id=835b5a2a-4d35-4d2a-9fdd-0c78b7def163,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap835b5a2a-4d')
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:30:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1028113698',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1028113698',id=9,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS3ZbVHfxQVFHpZS5QTkoqKDiEjmhCdyP6+fgfZz0jwXQ4fAjcP60l9wC1gyScpF8Zw9rmHw2YJpeRpTUHuz2Hroka/B8qwVo0YxuHbLGdfKbP5YGwIEhUARrat7zqI4A==',key_name='tempest-keypair-476892975',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:31:36Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='72d31bafe182459aa8bcb4f1bbb5e6d0',ramdisk_id='',reservation_id='r-b6ok04uz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-294347777',owner_user_name='tempest-TaggedBootDevicesTest_v242-294347777-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:31:36Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e7aeea82653d4681aa22f6dd8ae742b4',uuid=7119b8b3-291b-4868-820e-d7bd3dd3ada4,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "a1aaff41-e246-4887-89c6-a91163d4efd5", "address": "fa:16:3e:d2:e8:92", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.42", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa1aaff41-e2", "ovs_interfaceid": "a1aaff41-e246-4887-89c6-a91163d4efd5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converting VIF {"id": "a1aaff41-e246-4887-89c6-a91163d4efd5", "address": "fa:16:3e:d2:e8:92", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.42", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa1aaff41-e2", "ovs_interfaceid": "a1aaff41-e246-4887-89c6-a91163d4efd5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d2:e8:92,bridge_name='br-int',has_traffic_filtering=True,id=a1aaff41-e246-4887-89c6-a91163d4efd5,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapa1aaff41-e2') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG os_vif [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:d2:e8:92,bridge_name='br-int',has_traffic_filtering=True,id=a1aaff41-e246-4887-89c6-a91163d4efd5,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapa1aaff41-e2') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapa1aaff41-e2, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:52 user nova-compute[72081]: INFO os_vif [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:d2:e8:92,bridge_name='br-int',has_traffic_filtering=True,id=a1aaff41-e246-4887-89c6-a91163d4efd5,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapa1aaff41-e2')
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:30:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1028113698',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1028113698',id=9,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS3ZbVHfxQVFHpZS5QTkoqKDiEjmhCdyP6+fgfZz0jwXQ4fAjcP60l9wC1gyScpF8Zw9rmHw2YJpeRpTUHuz2Hroka/B8qwVo0YxuHbLGdfKbP5YGwIEhUARrat7zqI4A==',key_name='tempest-keypair-476892975',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:31:36Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='72d31bafe182459aa8bcb4f1bbb5e6d0',ramdisk_id='',reservation_id='r-b6ok04uz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-294347777',owner_user_name='tempest-TaggedBootDevicesTest_v242-294347777-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:31:36Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e7aeea82653d4681aa22f6dd8ae742b4',uuid=7119b8b3-291b-4868-820e-d7bd3dd3ada4,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "ac401bfa-a662-4290-af6e-0a39e8fde61c", "address": "fa:16:3e:db:3e:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.161", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapac401bfa-a6", "ovs_interfaceid": "ac401bfa-a662-4290-af6e-0a39e8fde61c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converting VIF {"id": "ac401bfa-a662-4290-af6e-0a39e8fde61c", "address": "fa:16:3e:db:3e:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.161", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapac401bfa-a6", "ovs_interfaceid": "ac401bfa-a662-4290-af6e-0a39e8fde61c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:db:3e:90,bridge_name='br-int',has_traffic_filtering=True,id=ac401bfa-a662-4290-af6e-0a39e8fde61c,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapac401bfa-a6') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG os_vif [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:db:3e:90,bridge_name='br-int',has_traffic_filtering=True,id=ac401bfa-a662-4290-af6e-0a39e8fde61c,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapac401bfa-a6') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapac401bfa-a6, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:52 user nova-compute[72081]: INFO os_vif [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:db:3e:90,bridge_name='br-int',has_traffic_filtering=True,id=ac401bfa-a662-4290-af6e-0a39e8fde61c,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapac401bfa-a6')
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:30:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1028113698',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1028113698',id=9,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS3ZbVHfxQVFHpZS5QTkoqKDiEjmhCdyP6+fgfZz0jwXQ4fAjcP60l9wC1gyScpF8Zw9rmHw2YJpeRpTUHuz2Hroka/B8qwVo0YxuHbLGdfKbP5YGwIEhUARrat7zqI4A==',key_name='tempest-keypair-476892975',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:31:36Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='72d31bafe182459aa8bcb4f1bbb5e6d0',ramdisk_id='',reservation_id='r-b6ok04uz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-294347777',owner_user_name='tempest-TaggedBootDevicesTest_v242-294347777-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:31:36Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e7aeea82653d4681aa22f6dd8ae742b4',uuid=7119b8b3-291b-4868-820e-d7bd3dd3ada4,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "4e57de22-e876-4bfe-bd07-a1003afd2295", "address": "fa:16:3e:93:42:94", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4e57de22-e8", "ovs_interfaceid": "4e57de22-e876-4bfe-bd07-a1003afd2295", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converting VIF {"id": "4e57de22-e876-4bfe-bd07-a1003afd2295", "address": "fa:16:3e:93:42:94", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4e57de22-e8", "ovs_interfaceid": "4e57de22-e876-4bfe-bd07-a1003afd2295", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:93:42:94,bridge_name='br-int',has_traffic_filtering=True,id=4e57de22-e876-4bfe-bd07-a1003afd2295,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4e57de22-e8') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG os_vif [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:93:42:94,bridge_name='br-int',has_traffic_filtering=True,id=4e57de22-e876-4bfe-bd07-a1003afd2295,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4e57de22-e8') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap4e57de22-e8, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:52 user nova-compute[72081]: INFO os_vif [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:93:42:94,bridge_name='br-int',has_traffic_filtering=True,id=4e57de22-e876-4bfe-bd07-a1003afd2295,network=Network(d4fb95ab-11b5-4eff-b746-035d8a799faa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4e57de22-e8')
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:30:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1028113698',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1028113698',id=9,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS3ZbVHfxQVFHpZS5QTkoqKDiEjmhCdyP6+fgfZz0jwXQ4fAjcP60l9wC1gyScpF8Zw9rmHw2YJpeRpTUHuz2Hroka/B8qwVo0YxuHbLGdfKbP5YGwIEhUARrat7zqI4A==',key_name='tempest-keypair-476892975',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:31:36Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='72d31bafe182459aa8bcb4f1bbb5e6d0',ramdisk_id='',reservation_id='r-b6ok04uz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-294347777',owner_user_name='tempest-TaggedBootDevicesTest_v242-294347777-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:31:36Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e7aeea82653d4681aa22f6dd8ae742b4',uuid=7119b8b3-291b-4868-820e-d7bd3dd3ada4,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "8248913f-46c8-4f3d-ab47-0203cb267169", "address": "fa:16:3e:21:f0:77", "network": {"id": "435f9638-5e8f-40a8-b271-a183982c70d5", "bridge": "br-int", "label": "tempest-device-tagging-net2-1738895025", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8248913f-46", "ovs_interfaceid": "8248913f-46c8-4f3d-ab47-0203cb267169", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converting VIF {"id": "8248913f-46c8-4f3d-ab47-0203cb267169", "address": "fa:16:3e:21:f0:77", "network": {"id": "435f9638-5e8f-40a8-b271-a183982c70d5", "bridge": "br-int", "label": "tempest-device-tagging-net2-1738895025", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8248913f-46", "ovs_interfaceid": "8248913f-46c8-4f3d-ab47-0203cb267169", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:21:f0:77,bridge_name='br-int',has_traffic_filtering=True,id=8248913f-46c8-4f3d-ab47-0203cb267169,network=Network(435f9638-5e8f-40a8-b271-a183982c70d5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8248913f-46') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG os_vif [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:21:f0:77,bridge_name='br-int',has_traffic_filtering=True,id=8248913f-46c8-4f3d-ab47-0203cb267169,network=Network(435f9638-5e8f-40a8-b271-a183982c70d5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8248913f-46') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap8248913f-46, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:52 user nova-compute[72081]: INFO os_vif [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:21:f0:77,bridge_name='br-int',has_traffic_filtering=True,id=8248913f-46c8-4f3d-ab47-0203cb267169,network=Network(435f9638-5e8f-40a8-b271-a183982c70d5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8248913f-46')
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:30:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1028113698',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1028113698',id=9,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS3ZbVHfxQVFHpZS5QTkoqKDiEjmhCdyP6+fgfZz0jwXQ4fAjcP60l9wC1gyScpF8Zw9rmHw2YJpeRpTUHuz2Hroka/B8qwVo0YxuHbLGdfKbP5YGwIEhUARrat7zqI4A==',key_name='tempest-keypair-476892975',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:31:36Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='72d31bafe182459aa8bcb4f1bbb5e6d0',ramdisk_id='',reservation_id='r-b6ok04uz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-294347777',owner_user_name='tempest-TaggedBootDevicesTest_v242-294347777-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:31:36Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e7aeea82653d4681aa22f6dd8ae742b4',uuid=7119b8b3-291b-4868-820e-d7bd3dd3ada4,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "21860860-2a08-43dc-bda8-0a1d0fab0634", "address": "fa:16:3e:eb:f1:cc", "network": {"id": "435f9638-5e8f-40a8-b271-a183982c70d5", "bridge": "br-int", "label": "tempest-device-tagging-net2-1738895025", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap21860860-2a", "ovs_interfaceid": "21860860-2a08-43dc-bda8-0a1d0fab0634", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converting VIF {"id": "21860860-2a08-43dc-bda8-0a1d0fab0634", "address": "fa:16:3e:eb:f1:cc", "network": {"id": "435f9638-5e8f-40a8-b271-a183982c70d5", "bridge": "br-int", "label": "tempest-device-tagging-net2-1738895025", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap21860860-2a", "ovs_interfaceid": "21860860-2a08-43dc-bda8-0a1d0fab0634", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:eb:f1:cc,bridge_name='br-int',has_traffic_filtering=True,id=21860860-2a08-43dc-bda8-0a1d0fab0634,network=Network(435f9638-5e8f-40a8-b271-a183982c70d5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap21860860-2a') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG os_vif [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:eb:f1:cc,bridge_name='br-int',has_traffic_filtering=True,id=21860860-2a08-43dc-bda8-0a1d0fab0634,network=Network(435f9638-5e8f-40a8-b271-a183982c70d5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap21860860-2a') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap21860860-2a, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:32:52 user nova-compute[72081]: INFO os_vif [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:eb:f1:cc,bridge_name='br-int',has_traffic_filtering=True,id=21860860-2a08-43dc-bda8-0a1d0fab0634,network=Network(435f9638-5e8f-40a8-b271-a183982c70d5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap21860860-2a')
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kdQHngfN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:32:52 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:32:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'drGIVBHg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:32:52 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:32:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.428s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] <== connect_volume: return (1431ms) {'type': 'block', 'scsi_wwn': '23263376262653465', 'path': '/dev/sdh'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23263376262653465', 'path': '/dev/sdh'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.257s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.objects.instance [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Lazy-loading 'flavor' on Instance uuid ccf9f424-d230-4079-87c5-bb08bc85b0b7 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[552f27df-9193-4e07-92ed-3c7caeef21c1]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:32:52 user nova-compute[72081]: No targets found.
Apr 12 03:32:52 user nova-compute[72081]: iSNS:
Apr 12 03:32:52 user nova-compute[72081]: No targets found.
Apr 12 03:32:52 user nova-compute[72081]: STATIC:
Apr 12 03:32:52 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550
Apr 12 03:32:52 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:52 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:52 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9
Apr 12 03:32:52 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:52 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:52 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1
Apr 12 03:32:52 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:52 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:52 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3
Apr 12 03:32:52 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:52 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:52 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3
Apr 12 03:32:52 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:52 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:52 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d
Apr 12 03:32:52 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:52 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:52 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd
Apr 12 03:32:52 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:52 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:52 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352
Apr 12 03:32:52 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:52 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:52 user nova-compute[72081]: FIRMWARE:
Apr 12 03:32:52 user nova-compute[72081]: No targets found.
Apr 12 03:32:52 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:32:52 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:32:52 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.compute.manager [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-plugged-ce9193f1-28a9-4c0f-95ae-e21f8140de19 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.compute.manager [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-plugged-ce9193f1-28a9-4c0f-95ae-e21f8140de19 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:52 user nova-compute[72081]: WARNING nova.compute.manager [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received unexpected event network-vif-plugged-ce9193f1-28a9-4c0f-95ae-e21f8140de19 for instance with vm_state active and task_state deleting.
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.compute.manager [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-unplugged-835b5a2a-4d35-4d2a-9fdd-0c78b7def163 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.compute.manager [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-unplugged-835b5a2a-4d35-4d2a-9fdd-0c78b7def163 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.compute.manager [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-unplugged-835b5a2a-4d35-4d2a-9fdd-0c78b7def163 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.compute.manager [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-plugged-835b5a2a-4d35-4d2a-9fdd-0c78b7def163 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.compute.manager [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-plugged-835b5a2a-4d35-4d2a-9fdd-0c78b7def163 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:52 user nova-compute[72081]: WARNING nova.compute.manager [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received unexpected event network-vif-plugged-835b5a2a-4d35-4d2a-9fdd-0c78b7def163 for instance with vm_state active and task_state deleting.
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.compute.manager [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-unplugged-4e57de22-e876-4bfe-bd07-a1003afd2295 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.compute.manager [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-unplugged-4e57de22-e876-4bfe-bd07-a1003afd2295 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.compute.manager [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-unplugged-4e57de22-e876-4bfe-bd07-a1003afd2295 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.compute.manager [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-plugged-4e57de22-e876-4bfe-bd07-a1003afd2295 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.compute.manager [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-plugged-4e57de22-e876-4bfe-bd07-a1003afd2295 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:52 user nova-compute[72081]: WARNING nova.compute.manager [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received unexpected event network-vif-plugged-4e57de22-e876-4bfe-bd07-a1003afd2295 for instance with vm_state active and task_state deleting.
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.compute.manager [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-unplugged-21860860-2a08-43dc-bda8-0a1d0fab0634 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.compute.manager [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-unplugged-21860860-2a08-43dc-bda8-0a1d0fab0634 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.compute.manager [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-unplugged-21860860-2a08-43dc-bda8-0a1d0fab0634 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.compute.manager [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-plugged-21860860-2a08-43dc-bda8-0a1d0fab0634 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG nova.compute.manager [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-plugged-21860860-2a08-43dc-bda8-0a1d0fab0634 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:52 user nova-compute[72081]: WARNING nova.compute.manager [req-1a389d3e-5dd0-4eea-a1c1-103a997384e7 req-b9aec0b3-0483-44c6-89f2-dbc27bbb29dd service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received unexpected event network-vif-plugged-21860860-2a08-43dc-bda8-0a1d0fab0634 for instance with vm_state active and task_state deleting.
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9edd84ef-3e91-42e8-9024-4d5f995f320f]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.061s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[073a38ce-833a-4b90-afbd-9cb5604d35b6]: (4, ('tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)\ntcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)
Apr 12 03:32:52 user nova-compute[72081]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:52 user nova-compute[72081]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:52 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:52 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:52 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:52 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:52 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:52 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsi session list stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)
Apr 12 03:32:52 user nova-compute[72081]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:52 user nova-compute[72081]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 (non-flash)
Apr 12 03:32:52 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:52 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:52 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:52 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:52 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:52 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fb41e0c10>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352'): ({'sdb'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Removing single pathed devices sdb {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c79316e3-901a-42d6-a346-bb49ecc18fde]: (4, ('path checker states:\nup                  7\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5224286-d3d7-4199-8ee9-d97a9bf3a32b]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Flushing IO for device /dev/sdb {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:32:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.012s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5a3ad65a-aff3-48eb-87fe-e6ea64ff2fd4]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-6a0cc91e-b8d3-4139-bb9f-fcef19e24337 req-85107d92-2602-4adf-b31a-fc2b8b7d459a service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-plugged-a1aaff41-e246-4887-89c6-a91163d4efd5 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6a0cc91e-b8d3-4139-bb9f-fcef19e24337 req-85107d92-2602-4adf-b31a-fc2b8b7d459a service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6a0cc91e-b8d3-4139-bb9f-fcef19e24337 req-85107d92-2602-4adf-b31a-fc2b8b7d459a service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6a0cc91e-b8d3-4139-bb9f-fcef19e24337 req-85107d92-2602-4adf-b31a-fc2b8b7d459a service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.004s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-6a0cc91e-b8d3-4139-bb9f-fcef19e24337 req-85107d92-2602-4adf-b31a-fc2b8b7d459a service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-plugged-a1aaff41-e246-4887-89c6-a91163d4efd5 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:53 user nova-compute[72081]: WARNING nova.compute.manager [req-6a0cc91e-b8d3-4139-bb9f-fcef19e24337 req-85107d92-2602-4adf-b31a-fc2b8b7d459a service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received unexpected event network-vif-plugged-a1aaff41-e246-4887-89c6-a91163d4efd5 for instance with vm_state active and task_state deleting.
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.064s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[eb88ddc7-36eb-4046-836b-3b268ec42586]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] SCSI volumes sdb have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d78c1666-426c-46f5-81f7-09cb08ac1117]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-26b842cd-51b1-4af6-808f-1f3e9197d8aa req-a2be10ab-11e0-4e1a-a780-0dcb41cf52f3 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-plugged-ac401bfa-a662-4290-af6e-0a39e8fde61c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-26b842cd-51b1-4af6-808f-1f3e9197d8aa req-a2be10ab-11e0-4e1a-a780-0dcb41cf52f3 service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-26b842cd-51b1-4af6-808f-1f3e9197d8aa req-a2be10ab-11e0-4e1a-a780-0dcb41cf52f3 service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-26b842cd-51b1-4af6-808f-1f3e9197d8aa req-a2be10ab-11e0-4e1a-a780-0dcb41cf52f3 service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-26b842cd-51b1-4af6-808f-1f3e9197d8aa req-a2be10ab-11e0-4e1a-a780-0dcb41cf52f3 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-plugged-ac401bfa-a662-4290-af6e-0a39e8fde61c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:53 user nova-compute[72081]: WARNING nova.compute.manager [req-26b842cd-51b1-4af6-808f-1f3e9197d8aa req-a2be10ab-11e0-4e1a-a780-0dcb41cf52f3 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received unexpected event network-vif-plugged-ac401bfa-a662-4290-af6e-0a39e8fde61c for instance with vm_state active and task_state deleting.
Apr 12 03:32:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-26b842cd-51b1-4af6-808f-1f3e9197d8aa req-a2be10ab-11e0-4e1a-a780-0dcb41cf52f3 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-unplugged-8248913f-46c8-4f3d-ab47-0203cb267169 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-26b842cd-51b1-4af6-808f-1f3e9197d8aa req-a2be10ab-11e0-4e1a-a780-0dcb41cf52f3 service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-26b842cd-51b1-4af6-808f-1f3e9197d8aa req-a2be10ab-11e0-4e1a-a780-0dcb41cf52f3 service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-26b842cd-51b1-4af6-808f-1f3e9197d8aa req-a2be10ab-11e0-4e1a-a780-0dcb41cf52f3 service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-26b842cd-51b1-4af6-808f-1f3e9197d8aa req-a2be10ab-11e0-4e1a-a780-0dcb41cf52f3 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-unplugged-8248913f-46c8-4f3d-ab47-0203cb267169 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-26b842cd-51b1-4af6-808f-1f3e9197d8aa req-a2be10ab-11e0-4e1a-a780-0dcb41cf52f3 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-unplugged-8248913f-46c8-4f3d-ab47-0203cb267169 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-26b842cd-51b1-4af6-808f-1f3e9197d8aa req-a2be10ab-11e0-4e1a-a780-0dcb41cf52f3 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-plugged-8248913f-46c8-4f3d-ab47-0203cb267169 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-26b842cd-51b1-4af6-808f-1f3e9197d8aa req-a2be10ab-11e0-4e1a-a780-0dcb41cf52f3 service nova] Acquiring lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-26b842cd-51b1-4af6-808f-1f3e9197d8aa req-a2be10ab-11e0-4e1a-a780-0dcb41cf52f3 service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-26b842cd-51b1-4af6-808f-1f3e9197d8aa req-a2be10ab-11e0-4e1a-a780-0dcb41cf52f3 service nova] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-26b842cd-51b1-4af6-808f-1f3e9197d8aa req-a2be10ab-11e0-4e1a-a780-0dcb41cf52f3 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] No waiting events found dispatching network-vif-plugged-8248913f-46c8-4f3d-ab47-0203cb267169 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:53 user nova-compute[72081]: WARNING nova.compute.manager [req-26b842cd-51b1-4af6-808f-1f3e9197d8aa req-a2be10ab-11e0-4e1a-a780-0dcb41cf52f3 service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received unexpected event network-vif-plugged-8248913f-46c8-4f3d-ab47-0203cb267169 for instance with vm_state active and task_state deleting.
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 -p 172.16.0.220:3260 --logout" returned: 0 in 0.078s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a1ad08b-4488-4529-b19e-22077f0b4a61]: (4, ('Logging out of session [sid: 2, target: iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352, portal: 172.16.0.220,3260]\nLogout of [sid: 2, target: iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 2, target: iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352, portal: 172.16.0.220,3260]
Apr 12 03:32:53 user nova-compute[72081]: Logout of [sid: 2, target: iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352, portal: 172.16.0.220,3260] successful.
Apr 12 03:32:53 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba2eb790-6032-4455-82ed-af81c61ca352 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.040s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bc20e8d3-3bed-4f4f-b607-98a97bffe45b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.437s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] <== disconnect_volume: return (693ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.463s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:32:53 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:32:53 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'POhliZru', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:32:53 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 -p 172.16.0.220:3260" returned: 21 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[896e9b2c-c57d-41d7-a8d8-33d6014bc282]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:32:53 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ea9e7baf-afd9-4306-89ec-c6c5f0e9e745]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93] added
Apr 12 03:32:53 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[626e3c2a-dd9a-44c2-9172-5ba688fa20c9]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d627d25a-5a78-4778-b879-a171b590d451]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v drGIVBHg {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] COPY block job progress, current cursor: 14680064 final cursor: 1073741824 {{(pid=72081) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v drGIVBHg" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3f9a67bd-0272-4714-a4f2-98b8a41d5ade]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'drGIVBHg'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f4db6ab6-3627-4aec-9a81-594f5edb90e5]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.034s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[88696ece-3e51-421a-af69-6cd7defd1521]: (4, ('tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)\ntcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:53 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsi session list stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:53 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 -p 172.16.0.220:3260 --login" returned: 0 in 0.095s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0d5f9f8d-321f-4807-bf0a-b874234b712c]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93, portal: 172.16.0.220,3260]
Apr 12 03:32:53 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93, portal: 172.16.0.220,3260] successful.
Apr 12 03:32:53 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.028s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b613ba34-92fe-451f-972f-36e3d42899ae]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.046s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[878fd411-6cc8-40d0-85da-a7655fed177b]: (4, ('tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)\ntcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)\ntcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:53 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsi session list stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:53 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:53 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.400s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] HCTL ('33', '-', '-', 0) found on session 13 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[da49a7ac-5838-4d41-8c3f-51933dca6034]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Searching for a device in session 13 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'drGIVBHg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:32:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] COPY block job progress, current cursor: 174063616 final cursor: 1073741824 {{(pid=72081) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] COPY block job progress, current cursor: 351272960 final cursor: 1073741824 {{(pid=72081) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.463s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] <== connect_volume: return (1928ms) {'type': 'block', 'scsi_wwn': '26631643561666465', 'path': '/dev/sdb'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26631643561666465', 'path': '/dev/sdb'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 1.461s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lazy-loading 'flavor' on Instance uuid cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6e6b6b9-ed08-4153-87a5-2069c080817b]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:32:54 user nova-compute[72081]: No targets found.
Apr 12 03:32:54 user nova-compute[72081]: iSNS:
Apr 12 03:32:54 user nova-compute[72081]: No targets found.
Apr 12 03:32:54 user nova-compute[72081]: STATIC:
Apr 12 03:32:54 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550
Apr 12 03:32:54 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:54 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:54 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93
Apr 12 03:32:54 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:54 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:54 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9
Apr 12 03:32:54 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:54 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:54 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1
Apr 12 03:32:54 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:54 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:54 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3
Apr 12 03:32:54 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:54 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:54 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3
Apr 12 03:32:54 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:54 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:54 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d
Apr 12 03:32:54 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:54 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:54 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd
Apr 12 03:32:54 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:54 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:54 user nova-compute[72081]: FIRMWARE:
Apr 12 03:32:54 user nova-compute[72081]: No targets found.
Apr 12 03:32:54 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:32:54 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:32:54 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] attach device xml: <disk type="block" device="disk">
Apr 12 03:32:54 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:32:54 user nova-compute[72081]:   <source dev="/dev/sdb"/>
Apr 12 03:32:54 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:32:54 user nova-compute[72081]:   <serial>263426d6-d387-4a67-9013-5a23e8386c93</serial>
Apr 12 03:32:54 user nova-compute[72081]: </disk>
Apr 12 03:32:54 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.029s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[df77180e-eb13-4773-9d77-805204b07da5]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.045s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[fde558e0-8f3d-4309-ae4f-8c00b011661a]: (4, ('tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)\ntcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)\ntcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)
Apr 12 03:32:54 user nova-compute[72081]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:54 user nova-compute[72081]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)
Apr 12 03:32:54 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:54 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:54 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:54 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:54 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:54 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsi session list stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)
Apr 12 03:32:54 user nova-compute[72081]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:54 user nova-compute[72081]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)
Apr 12 03:32:54 user nova-compute[72081]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd (non-flash)
Apr 12 03:32:54 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:54 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:54 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:54 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:54 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fac35eb00>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd'): ({'sdc'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Removing single pathed devices sdc {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.029s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a98bcbeb-892d-414a-bd73-b92634aa39e0]: (4, ('path checker states:\nup                  8\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1af8338a-d45b-4e8f-a6f9-40c00253a321]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Flushing IO for device /dev/sdc {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] COPY block job progress, current cursor: 506462208 final cursor: 1073741824 {{(pid=72081) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[68acb53d-9197-4899-a259-84f0a78f606f]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.036s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[180b524a-34c0-482d-a0e8-fa2851c20946]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] SCSI volumes sdc have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[46112a43-ae58-4908-99db-41498f00bf44]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:54 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd -p 172.16.0.220:3260 --logout" returned: 0 in 0.083s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[dd155ef3-d25c-468a-acc6-0366b40168f7]: (4, ('Logging out of session [sid: 3, target: iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd, portal: 172.16.0.220,3260]\nLogout of [sid: 3, target: iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 3, target: iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd, portal: 172.16.0.220,3260]
Apr 12 03:32:55 user nova-compute[72081]: Logout of [sid: 3, target: iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd, portal: 172.16.0.220,3260] successful.
Apr 12 03:32:55 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:19d5feb6-1f29-4d61-8901-3b445f0c26fd -p 172.16.0.220:3260 --op delete" returned: 0 in 0.043s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5721687b-57aa-4f97-a101-22807bca706e]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.422s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] <== disconnect_volume: return (1883ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'PBUOnthu', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:32:55 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:32:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.028s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9a200e26-f00c-4300-900f-4a988d277778]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:32:55 user nova-compute[72081]: No targets found.
Apr 12 03:32:55 user nova-compute[72081]: iSNS:
Apr 12 03:32:55 user nova-compute[72081]: No targets found.
Apr 12 03:32:55 user nova-compute[72081]: STATIC:
Apr 12 03:32:55 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550
Apr 12 03:32:55 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:55 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:55 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93
Apr 12 03:32:55 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:55 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:55 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9
Apr 12 03:32:55 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:55 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:55 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1
Apr 12 03:32:55 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:55 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:55 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3
Apr 12 03:32:55 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:55 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:55 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3
Apr 12 03:32:55 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:55 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:55 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d
Apr 12 03:32:55 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:55 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:55 user nova-compute[72081]: FIRMWARE:
Apr 12 03:32:55 user nova-compute[72081]: No targets found.
Apr 12 03:32:55 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:32:55 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:32:55 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1ac68df6-2716-46f9-9bf6-40db3349aa26]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG nova.compute.manager [req-b921dc4e-0bd3-4cc8-89fb-8492048f57d5 req-40f38e12-2494-452f-a5f6-d37423f96719 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-changed-33b2e7fb-520f-46d4-be86-e492aff77d4b {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG nova.compute.manager [req-b921dc4e-0bd3-4cc8-89fb-8492048f57d5 req-40f38e12-2494-452f-a5f6-d37423f96719 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Refreshing instance network info cache due to event network-changed-33b2e7fb-520f-46d4-be86-e492aff77d4b. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b921dc4e-0bd3-4cc8-89fb-8492048f57d5 req-40f38e12-2494-452f-a5f6-d37423f96719 service nova] Acquiring lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b921dc4e-0bd3-4cc8-89fb-8492048f57d5 req-40f38e12-2494-452f-a5f6-d37423f96719 service nova] Acquired lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG nova.network.neutron [req-b921dc4e-0bd3-4cc8-89fb-8492048f57d5 req-40f38e12-2494-452f-a5f6-d37423f96719 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Refreshing network info cache for port 33b2e7fb-520f-46d4-be86-e492aff77d4b {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.035s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8e3f665f-842e-48f7-8ad8-c6d1b16ad80e]: (4, ('tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)\ntcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)\ntcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)
Apr 12 03:32:55 user nova-compute[72081]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:55 user nova-compute[72081]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)
Apr 12 03:32:55 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:55 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:55 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:55 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:55 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsi session list stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)
Apr 12 03:32:55 user nova-compute[72081]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:55 user nova-compute[72081]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)
Apr 12 03:32:55 user nova-compute[72081]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d (non-flash)
Apr 12 03:32:55 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:55 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:55 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:55 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3f941f9240>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d'): ({'sdd'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Removing single pathed devices sdd {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] No VIF found with MAC fa:16:3e:a8:9f:cb, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e26610ce-8a4d-4f41-b876-e1dcfa78960a]: (4, ('path checker states:\nup                  7\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4a85e300-6a6a-4079-a2fc-c56447ce2621]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Flushing IO for device /dev/sdd {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d329de54-2171-4e07-80de-6616d5d57ba0]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] COPY block job progress, current cursor: 658505728 final cursor: 1073741824 {{(pid=72081) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.069s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0cdb3c7e-5e95-4fce-ada8-e9af63b76b02]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] SCSI volumes sdd have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[579028ab-d469-4399-b980-11e938109b37]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4b2fc291-f18f-4a89-959f-8659be17743f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 7.918s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d -p 172.16.0.220:3260 --logout" returned: 0 in 0.081s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3431b442-c334-4623-a08d-c9bf036d8d0c]: (4, ('Logging out of session [sid: 4, target: iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d, portal: 172.16.0.220,3260]\nLogout of [sid: 4, target: iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 4, target: iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d, portal: 172.16.0.220,3260]
Apr 12 03:32:55 user nova-compute[72081]: Logout of [sid: 4, target: iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d, portal: 172.16.0.220,3260] successful.
Apr 12 03:32:55 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c73ac339-e533-4c8b-958d-ddf3e59dfb7d -p 172.16.0.220:3260 --op delete" returned: 0 in 0.036s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[09639440-9b07-4edc-a88a-ddcb1237b91f]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.462s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] <== disconnect_volume: return (464ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:32:55 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Deleting instance files /opt/stack/data/nova/instances/7119b8b3-291b-4868-820e-d7bd3dd3ada4_del
Apr 12 03:32:55 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Deletion of /opt/stack/data/nova/instances/7119b8b3-291b-4868-820e-d7bd3dd3ada4_del complete
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:32:55 user nova-compute[72081]: INFO nova.compute.manager [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Took 5.31 seconds to destroy the instance on the hypervisor.
Apr 12 03:32:55 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:32:55 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] COPY block job progress, current cursor: 809500672 final cursor: 1073741824 {{(pid=72081) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 12 03:32:56 user nova-compute[72081]: DEBUG nova.network.neutron [req-b921dc4e-0bd3-4cc8-89fb-8492048f57d5 req-40f38e12-2494-452f-a5f6-d37423f96719 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Updated VIF entry in instance network info cache for port 33b2e7fb-520f-46d4-be86-e492aff77d4b. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:32:56 user nova-compute[72081]: DEBUG nova.network.neutron [req-b921dc4e-0bd3-4cc8-89fb-8492048f57d5 req-40f38e12-2494-452f-a5f6-d37423f96719 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Updating instance_info_cache with network_info: [{"id": "33b2e7fb-520f-46d4-be86-e492aff77d4b", "address": "fa:16:3e:f1:16:6c", "network": {"id": "0905eabd-7263-448c-97f6-296c5b42cd24", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-192357488-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.207", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap33b2e7fb-52", "ovs_interfaceid": "33b2e7fb-520f-46d4-be86-e492aff77d4b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "7740f806-8319-484b-8b0f-bffd8c5fe532", "address": "fa:16:3e:10:f4:89", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.24", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7740f806-83", "ovs_interfaceid": "7740f806-8319-484b-8b0f-bffd8c5fe532", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "be3de447-6d22-4fec-b962-0ebe772ed86f", "address": "fa:16:3e:15:de:ed", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.143", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe3de447-6d", "ovs_interfaceid": "be3de447-6d22-4fec-b962-0ebe772ed86f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "address": "fa:16:3e:38:3d:5e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.54", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap91177380-5d", "ovs_interfaceid": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "address": "fa:16:3e:02:8f:9e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.17", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap41626728-b2", "ovs_interfaceid": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "address": "fa:16:3e:f2:df:e5", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bb0c89d-0a", "ovs_interfaceid": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "address": "fa:16:3e:db:9a:17", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16cfa52-5c", "ovs_interfaceid": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:32:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b921dc4e-0bd3-4cc8-89fb-8492048f57d5 req-40f38e12-2494-452f-a5f6-d37423f96719 service nova] Releasing lock "refresh_cache-43289ac5-3300-4e1f-8d07-2d66ee78e7ba" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:32:56 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] COPY block job progress, current cursor: 975175680 final cursor: 1073741824 {{(pid=72081) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 12 03:32:56 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:32:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:56 user nova-compute[72081]: INFO nova.compute.manager [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Terminating instance
Apr 12 03:32:56 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:32:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:56 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] COPY block job progress, current cursor: 1073741824 final cursor: 1073741824 {{(pid=72081) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 12 03:32:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kdDrZQOh', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdj', 'volume_id': '9a82b8b7-d64b-4407-869f-d5b1dcdf0550': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:32:57 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:32:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.029s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d8b6fe6f-782c-4880-9c1f-6453c13d82c7]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:32:57 user nova-compute[72081]: No targets found.
Apr 12 03:32:57 user nova-compute[72081]: iSNS:
Apr 12 03:32:57 user nova-compute[72081]: No targets found.
Apr 12 03:32:57 user nova-compute[72081]: STATIC:
Apr 12 03:32:57 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550
Apr 12 03:32:57 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:57 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:57 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93
Apr 12 03:32:57 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:57 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:57 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9
Apr 12 03:32:57 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:57 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:57 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1
Apr 12 03:32:57 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:57 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:57 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3
Apr 12 03:32:57 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:57 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:57 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3
Apr 12 03:32:57 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:57 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:57 user nova-compute[72081]: FIRMWARE:
Apr 12 03:32:57 user nova-compute[72081]: No targets found.
Apr 12 03:32:57 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:32:57 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:32:57 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d93decf5-c759-471d-bbf9-a0d3780b1798]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG nova.compute.manager [req-e9fabb25-15f4-4d08-b3e8-49a8fbe72d15 req-ebea88c9-f1a6-4326-9a33-9956de8d3c31 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-unplugged-33b2e7fb-520f-46d4-be86-e492aff77d4b {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e9fabb25-15f4-4d08-b3e8-49a8fbe72d15 req-ebea88c9-f1a6-4326-9a33-9956de8d3c31 service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e9fabb25-15f4-4d08-b3e8-49a8fbe72d15 req-ebea88c9-f1a6-4326-9a33-9956de8d3c31 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e9fabb25-15f4-4d08-b3e8-49a8fbe72d15 req-ebea88c9-f1a6-4326-9a33-9956de8d3c31 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG nova.compute.manager [req-e9fabb25-15f4-4d08-b3e8-49a8fbe72d15 req-ebea88c9-f1a6-4326-9a33-9956de8d3c31 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-unplugged-33b2e7fb-520f-46d4-be86-e492aff77d4b {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG nova.compute.manager [req-e9fabb25-15f4-4d08-b3e8-49a8fbe72d15 req-ebea88c9-f1a6-4326-9a33-9956de8d3c31 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-unplugged-33b2e7fb-520f-46d4-be86-e492aff77d4b for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.042s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4ce2ec18-14a8-4a31-8c71-84457d93ddea]: (4, ('tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)\ntcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)\ntcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)
Apr 12 03:32:57 user nova-compute[72081]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:57 user nova-compute[72081]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)
Apr 12 03:32:57 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:57 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:57 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:57 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsi session list stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 (non-flash)
Apr 12 03:32:57 user nova-compute[72081]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:57 user nova-compute[72081]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)
Apr 12 03:32:57 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:57 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:57 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:57 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fb61af520>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550'): ({'sdj'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Removing single pathed devices sdj {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.028s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[649bfe2e-7de8-49cf-99bd-5dfc0f30fb21]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdj {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdj" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[16925c61-49e5-4b96-888a-41866452e691]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Flushing IO for device /dev/sdj {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdj {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdj" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7e312a7c-c1d0-473c-aab0-c7549d2b5dfb]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Remove SCSI device /dev/sdj with /sys/block/sdj/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdj/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG nova.compute.manager [req-2af93647-1628-45e6-a053-fb88713d94e1 req-6d76e04d-32cc-485e-8dfd-d24461e50f4d service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-unplugged-be3de447-6d22-4fec-b962-0ebe772ed86f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2af93647-1628-45e6-a053-fb88713d94e1 req-6d76e04d-32cc-485e-8dfd-d24461e50f4d service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2af93647-1628-45e6-a053-fb88713d94e1 req-6d76e04d-32cc-485e-8dfd-d24461e50f4d service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2af93647-1628-45e6-a053-fb88713d94e1 req-6d76e04d-32cc-485e-8dfd-d24461e50f4d service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG nova.compute.manager [req-2af93647-1628-45e6-a053-fb88713d94e1 req-6d76e04d-32cc-485e-8dfd-d24461e50f4d service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-unplugged-be3de447-6d22-4fec-b962-0ebe772ed86f {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG nova.compute.manager [req-2af93647-1628-45e6-a053-fb88713d94e1 req-6d76e04d-32cc-485e-8dfd-d24461e50f4d service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-unplugged-be3de447-6d22-4fec-b962-0ebe772ed86f for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdj/device/delete" returned: 0 in 0.055s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[cf025338-38d6-4b87-809c-6dd8f9479603]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Checking to see if SCSI volumes sdj have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] SCSI volumes sdj have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[86339967-a99c-448d-91f5-de5560ff06cc]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 -p 172.16.0.220:3260 --logout" returned: 0 in 0.100s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[595b2b55-4d05-42b5-9d86-01ed7aa7d99a]: (4, ('Logging out of session [sid: 11, target: iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550, portal: 172.16.0.220,3260]\nLogout of [sid: 11, target: iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('--logout',): stdout=Logging out of session [sid: 11, target: iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550, portal: 172.16.0.220,3260]
Apr 12 03:32:57 user nova-compute[72081]: Logout of [sid: 11, target: iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550, portal: 172.16.0.220,3260] successful.
Apr 12 03:32:57 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9a82b8b7-d64b-4407-869f-d5b1dcdf0550 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.032s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a60ce4f5-ed70-4879-854d-48f80403a224]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.447s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] <== disconnect_volume: return (449ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] swap_volume: Driver volume swap returned, new connection_info is now : {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wPNTeqDg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdh'}, 'status': 'reserved', 'instance': 'ccf9f424-d230-4079-87c5-bb08bc85b0b7', 'attached_at': '', 'detached_at': '', 'volume_id': '6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3', 'serial'} {{(pid=72081) _swap_volume /opt/stack/nova/nova/compute/manager.py:7665}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] swap_volume: removing Cinder connection for volume 9a82b8b7-d64b-4407-869f-d5b1dcdf0550 {{(pid=72081) _swap_volume /opt/stack/nova/nova/compute/manager.py:7710}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG nova.compute.manager [req-9b956faf-489d-4b34-8536-dec2a52ca186 req-bdadc14c-c0a6-45da-abd5-ebcef2b2fdb3 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-unplugged-41626728-b21d-4ef8-9d86-5328d2feb3f9 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9b956faf-489d-4b34-8536-dec2a52ca186 req-bdadc14c-c0a6-45da-abd5-ebcef2b2fdb3 service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9b956faf-489d-4b34-8536-dec2a52ca186 req-bdadc14c-c0a6-45da-abd5-ebcef2b2fdb3 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9b956faf-489d-4b34-8536-dec2a52ca186 req-bdadc14c-c0a6-45da-abd5-ebcef2b2fdb3 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.004s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG nova.compute.manager [req-9b956faf-489d-4b34-8536-dec2a52ca186 req-bdadc14c-c0a6-45da-abd5-ebcef2b2fdb3 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-unplugged-41626728-b21d-4ef8-9d86-5328d2feb3f9 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG nova.compute.manager [req-9b956faf-489d-4b34-8536-dec2a52ca186 req-bdadc14c-c0a6-45da-abd5-ebcef2b2fdb3 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-unplugged-41626728-b21d-4ef8-9d86-5328d2feb3f9 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:32:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681270363.3269129, 413850ca-8175-4ed6-a6cf-ff42230fa1e0 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:32:58 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] VM Stopped (Lifecycle Event)
Apr 12 03:32:58 user nova-compute[72081]: DEBUG nova.compute.manager [None req-149a2c55-539f-4b23-98bc-adf635fd2227 None None] [instance: 413850ca-8175-4ed6-a6cf-ff42230fa1e0] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:32:58 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Instance destroyed successfully.
Apr 12 03:32:58 user nova-compute[72081]: DEBUG nova.objects.instance [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lazy-loading 'resources' on Instance uuid 43289ac5-3300-4e1f-8d07-2d66ee78e7ba {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:31:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-780207909',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-780207909',id=10,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNCyWVMCVeXTj4eZLmBDk3uEr+ohaWpwIv6zTkIlNZ2QRH2THWN8SXei5drq2IJGQkIvQoNzOXVCuGCaf7h5mScjI1xfzmSUsrhlqHzEQ4MwAG4OF6CGlMBmuH+1KctDlA==',key_name='tempest-keypair-2115967448',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:31:40Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='bd808e7e237f4c7490abaff5db0e3529',ramdisk_id='',reservation_id='r-vmalcql5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-1194851633',owner_user_name='tempest-TaggedBootDevicesTest-1194851633-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:31:41Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d29b6ed431d64bcab02c2348feab9aa6',uuid=43289ac5-3300-4e1f-8d07-2d66ee78e7ba,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "33b2e7fb-520f-46d4-be86-e492aff77d4b", "address": "fa:16:3e:f1:16:6c", "network": {"id": "0905eabd-7263-448c-97f6-296c5b42cd24", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-192357488-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.207", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap33b2e7fb-52", "ovs_interfaceid": "33b2e7fb-520f-46d4-be86-e492aff77d4b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converting VIF {"id": "33b2e7fb-520f-46d4-be86-e492aff77d4b", "address": "fa:16:3e:f1:16:6c", "network": {"id": "0905eabd-7263-448c-97f6-296c5b42cd24", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-192357488-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.207", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap33b2e7fb-52", "ovs_interfaceid": "33b2e7fb-520f-46d4-be86-e492aff77d4b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:f1:16:6c,bridge_name='br-int',has_traffic_filtering=True,id=33b2e7fb-520f-46d4-be86-e492aff77d4b,network=Network(0905eabd-7263-448c-97f6-296c5b42cd24),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap33b2e7fb-52') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG os_vif [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:f1:16:6c,bridge_name='br-int',has_traffic_filtering=True,id=33b2e7fb-520f-46d4-be86-e492aff77d4b,network=Network(0905eabd-7263-448c-97f6-296c5b42cd24),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap33b2e7fb-52') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap33b2e7fb-52, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:58 user nova-compute[72081]: INFO os_vif [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:f1:16:6c,bridge_name='br-int',has_traffic_filtering=True,id=33b2e7fb-520f-46d4-be86-e492aff77d4b,network=Network(0905eabd-7263-448c-97f6-296c5b42cd24),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap33b2e7fb-52')
Apr 12 03:32:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:31:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-780207909',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-780207909',id=10,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNCyWVMCVeXTj4eZLmBDk3uEr+ohaWpwIv6zTkIlNZ2QRH2THWN8SXei5drq2IJGQkIvQoNzOXVCuGCaf7h5mScjI1xfzmSUsrhlqHzEQ4MwAG4OF6CGlMBmuH+1KctDlA==',key_name='tempest-keypair-2115967448',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:31:40Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='bd808e7e237f4c7490abaff5db0e3529',ramdisk_id='',reservation_id='r-vmalcql5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-1194851633',owner_user_name='tempest-TaggedBootDevicesTest-1194851633-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:31:41Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d29b6ed431d64bcab02c2348feab9aa6',uuid=43289ac5-3300-4e1f-8d07-2d66ee78e7ba,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "7740f806-8319-484b-8b0f-bffd8c5fe532", "address": "fa:16:3e:10:f4:89", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.24", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7740f806-83", "ovs_interfaceid": "7740f806-8319-484b-8b0f-bffd8c5fe532", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converting VIF {"id": "7740f806-8319-484b-8b0f-bffd8c5fe532", "address": "fa:16:3e:10:f4:89", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.24", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7740f806-83", "ovs_interfaceid": "7740f806-8319-484b-8b0f-bffd8c5fe532", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:10:f4:89,bridge_name='br-int',has_traffic_filtering=True,id=7740f806-8319-484b-8b0f-bffd8c5fe532,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap7740f806-83') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG os_vif [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:10:f4:89,bridge_name='br-int',has_traffic_filtering=True,id=7740f806-8319-484b-8b0f-bffd8c5fe532,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap7740f806-83') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap7740f806-83, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] swap_volume: Updating volume 9a82b8b7-d64b-4407-869f-d5b1dcdf0550 BDM record with {'connection_info': '{"driver_volume_type": "iscsi", "data": {"target_iqn": "iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3", "target_portal": "172.16.0.220:3260", "target_discovered": false, "auth_method": "CHAP", "auth_username": "wPNTeqDg", "auth_password": "UgihpOB0H59lDc", "target_lun": 0, "qos_specs": null, "access_mode": "rw", "encrypted": false, "cacheable": false, "device_path": "/dev/sdh", "volume_id": "6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3"}, "status": "reserved", "instance": "ccf9f424-d230-4079-87c5-bb08bc85b0b7", "attached_at": "", "detached_at": "", "volume_id": "6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3", "serial": "6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3"}', 'source_type': 'volume', 'destination_type': 'volume', 'snapshot_id': None, 'volume_id': '6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3', 'no_device': None, 'attachment_id': '6246ed7d-f657-4021-a573-3557302a99b8'} {{(pid=72081) _do_swap_volume /opt/stack/nova/nova/compute/manager.py:7858}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:58 user nova-compute[72081]: INFO os_vif [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:10:f4:89,bridge_name='br-int',has_traffic_filtering=True,id=7740f806-8319-484b-8b0f-bffd8c5fe532,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap7740f806-83')
Apr 12 03:32:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:31:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-780207909',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-780207909',id=10,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNCyWVMCVeXTj4eZLmBDk3uEr+ohaWpwIv6zTkIlNZ2QRH2THWN8SXei5drq2IJGQkIvQoNzOXVCuGCaf7h5mScjI1xfzmSUsrhlqHzEQ4MwAG4OF6CGlMBmuH+1KctDlA==',key_name='tempest-keypair-2115967448',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:31:40Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='bd808e7e237f4c7490abaff5db0e3529',ramdisk_id='',reservation_id='r-vmalcql5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-1194851633',owner_user_name='tempest-TaggedBootDevicesTest-1194851633-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:31:41Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d29b6ed431d64bcab02c2348feab9aa6',uuid=43289ac5-3300-4e1f-8d07-2d66ee78e7ba,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "be3de447-6d22-4fec-b962-0ebe772ed86f", "address": "fa:16:3e:15:de:ed", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.143", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe3de447-6d", "ovs_interfaceid": "be3de447-6d22-4fec-b962-0ebe772ed86f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converting VIF {"id": "be3de447-6d22-4fec-b962-0ebe772ed86f", "address": "fa:16:3e:15:de:ed", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.143", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe3de447-6d", "ovs_interfaceid": "be3de447-6d22-4fec-b962-0ebe772ed86f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:15:de:ed,bridge_name='br-int',has_traffic_filtering=True,id=be3de447-6d22-4fec-b962-0ebe772ed86f,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapbe3de447-6d') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG os_vif [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:15:de:ed,bridge_name='br-int',has_traffic_filtering=True,id=be3de447-6d22-4fec-b962-0ebe772ed86f,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapbe3de447-6d') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapbe3de447-6d, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ff962041-66d5-4114-a9de-5cd7619edb11 tempest-TestVolumeSwap-1083163114 tempest-TestVolumeSwap-1083163114-project-admin] Lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7" "released" by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" :: held 12.419s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:58 user nova-compute[72081]: INFO os_vif [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:15:de:ed,bridge_name='br-int',has_traffic_filtering=True,id=be3de447-6d22-4fec-b962-0ebe772ed86f,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapbe3de447-6d')
Apr 12 03:32:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:31:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-780207909',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-780207909',id=10,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNCyWVMCVeXTj4eZLmBDk3uEr+ohaWpwIv6zTkIlNZ2QRH2THWN8SXei5drq2IJGQkIvQoNzOXVCuGCaf7h5mScjI1xfzmSUsrhlqHzEQ4MwAG4OF6CGlMBmuH+1KctDlA==',key_name='tempest-keypair-2115967448',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:31:40Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='bd808e7e237f4c7490abaff5db0e3529',ramdisk_id='',reservation_id='r-vmalcql5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-1194851633',owner_user_name='tempest-TaggedBootDevicesTest-1194851633-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:31:41Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d29b6ed431d64bcab02c2348feab9aa6',uuid=43289ac5-3300-4e1f-8d07-2d66ee78e7ba,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "address": "fa:16:3e:38:3d:5e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.54", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap91177380-5d", "ovs_interfaceid": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converting VIF {"id": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "address": "fa:16:3e:38:3d:5e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.54", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap91177380-5d", "ovs_interfaceid": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:38:3d:5e,bridge_name='br-int',has_traffic_filtering=True,id=91177380-5d77-4bed-9ab0-e9085fcb3d5a,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap91177380-5d') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG os_vif [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:38:3d:5e,bridge_name='br-int',has_traffic_filtering=True,id=91177380-5d77-4bed-9ab0-e9085fcb3d5a,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap91177380-5d') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap91177380-5d, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:58 user nova-compute[72081]: INFO os_vif [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:38:3d:5e,bridge_name='br-int',has_traffic_filtering=True,id=91177380-5d77-4bed-9ab0-e9085fcb3d5a,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap91177380-5d')
Apr 12 03:32:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:31:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-780207909',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-780207909',id=10,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNCyWVMCVeXTj4eZLmBDk3uEr+ohaWpwIv6zTkIlNZ2QRH2THWN8SXei5drq2IJGQkIvQoNzOXVCuGCaf7h5mScjI1xfzmSUsrhlqHzEQ4MwAG4OF6CGlMBmuH+1KctDlA==',key_name='tempest-keypair-2115967448',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:31:40Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='bd808e7e237f4c7490abaff5db0e3529',ramdisk_id='',reservation_id='r-vmalcql5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-1194851633',owner_user_name='tempest-TaggedBootDevicesTest-1194851633-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:31:41Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d29b6ed431d64bcab02c2348feab9aa6',uuid=43289ac5-3300-4e1f-8d07-2d66ee78e7ba,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "address": "fa:16:3e:02:8f:9e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.17", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap41626728-b2", "ovs_interfaceid": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converting VIF {"id": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "address": "fa:16:3e:02:8f:9e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.17", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap41626728-b2", "ovs_interfaceid": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:02:8f:9e,bridge_name='br-int',has_traffic_filtering=True,id=41626728-b21d-4ef8-9d86-5328d2feb3f9,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap41626728-b2') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG os_vif [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:02:8f:9e,bridge_name='br-int',has_traffic_filtering=True,id=41626728-b21d-4ef8-9d86-5328d2feb3f9,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap41626728-b2') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap41626728-b2, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:59 user nova-compute[72081]: INFO os_vif [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:02:8f:9e,bridge_name='br-int',has_traffic_filtering=True,id=41626728-b21d-4ef8-9d86-5328d2feb3f9,network=Network(6274e1ca-390b-4378-9386-d41818b17cb3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap41626728-b2')
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:31:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-780207909',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-780207909',id=10,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNCyWVMCVeXTj4eZLmBDk3uEr+ohaWpwIv6zTkIlNZ2QRH2THWN8SXei5drq2IJGQkIvQoNzOXVCuGCaf7h5mScjI1xfzmSUsrhlqHzEQ4MwAG4OF6CGlMBmuH+1KctDlA==',key_name='tempest-keypair-2115967448',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:31:40Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='bd808e7e237f4c7490abaff5db0e3529',ramdisk_id='',reservation_id='r-vmalcql5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-1194851633',owner_user_name='tempest-TaggedBootDevicesTest-1194851633-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:31:41Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d29b6ed431d64bcab02c2348feab9aa6',uuid=43289ac5-3300-4e1f-8d07-2d66ee78e7ba,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "address": "fa:16:3e:f2:df:e5", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bb0c89d-0a", "ovs_interfaceid": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converting VIF {"id": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "address": "fa:16:3e:f2:df:e5", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bb0c89d-0a", "ovs_interfaceid": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f2:df:e5,bridge_name='br-int',has_traffic_filtering=True,id=6bb0c89d-0a60-4413-b09d-08d4d6ea3a52,network=Network(2767acc3-5e96-41ad-a7f4-fbd5b4acce3a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6bb0c89d-0a') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_vif [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f2:df:e5,bridge_name='br-int',has_traffic_filtering=True,id=6bb0c89d-0a60-4413-b09d-08d4d6ea3a52,network=Network(2767acc3-5e96-41ad-a7f4-fbd5b4acce3a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6bb0c89d-0a') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap6bb0c89d-0a, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:59 user nova-compute[72081]: INFO os_vif [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f2:df:e5,bridge_name='br-int',has_traffic_filtering=True,id=6bb0c89d-0a60-4413-b09d-08d4d6ea3a52,network=Network(2767acc3-5e96-41ad-a7f4-fbd5b4acce3a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6bb0c89d-0a')
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:31:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-780207909',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-780207909',id=10,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNCyWVMCVeXTj4eZLmBDk3uEr+ohaWpwIv6zTkIlNZ2QRH2THWN8SXei5drq2IJGQkIvQoNzOXVCuGCaf7h5mScjI1xfzmSUsrhlqHzEQ4MwAG4OF6CGlMBmuH+1KctDlA==',key_name='tempest-keypair-2115967448',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:31:40Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='bd808e7e237f4c7490abaff5db0e3529',ramdisk_id='',reservation_id='r-vmalcql5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-1194851633',owner_user_name='tempest-TaggedBootDevicesTest-1194851633-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:31:41Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d29b6ed431d64bcab02c2348feab9aa6',uuid=43289ac5-3300-4e1f-8d07-2d66ee78e7ba,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "address": "fa:16:3e:db:9a:17", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16cfa52-5c", "ovs_interfaceid": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converting VIF {"id": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "address": "fa:16:3e:db:9a:17", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16cfa52-5c", "ovs_interfaceid": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:db:9a:17,bridge_name='br-int',has_traffic_filtering=True,id=e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1,network=Network(2767acc3-5e96-41ad-a7f4-fbd5b4acce3a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16cfa52-5c') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_vif [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:db:9a:17,bridge_name='br-int',has_traffic_filtering=True,id=e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1,network=Network(2767acc3-5e96-41ad-a7f4-fbd5b4acce3a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16cfa52-5c') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tape16cfa52-5c, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:32:59 user nova-compute[72081]: INFO os_vif [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:db:9a:17,bridge_name='br-int',has_traffic_filtering=True,id=e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1,network=Network(2767acc3-5e96-41ad-a7f4-fbd5b4acce3a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16cfa52-5c')
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CmWAbLhi', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:32:59 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f39978e-ba0c-4e4a-af32-d1a7086d1a86]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:32:59 user nova-compute[72081]: No targets found.
Apr 12 03:32:59 user nova-compute[72081]: iSNS:
Apr 12 03:32:59 user nova-compute[72081]: No targets found.
Apr 12 03:32:59 user nova-compute[72081]: STATIC:
Apr 12 03:32:59 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93
Apr 12 03:32:59 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:59 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:59 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9
Apr 12 03:32:59 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:59 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:59 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1
Apr 12 03:32:59 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:59 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:59 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3
Apr 12 03:32:59 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:59 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:59 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3
Apr 12 03:32:59 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:59 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:59 user nova-compute[72081]: FIRMWARE:
Apr 12 03:32:59 user nova-compute[72081]: No targets found.
Apr 12 03:32:59 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:32:59 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:32:59 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0fa1c872-df9a-4556-953d-f456296d97df]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[caef65b5-cc45-4bc8-a17f-a9d6954ac429]: (4, ('tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)\ntcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:59 user nova-compute[72081]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)
Apr 12 03:32:59 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:59 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:59 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:59 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsi session list stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:59 user nova-compute[72081]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)
Apr 12 03:32:59 user nova-compute[72081]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 (non-flash)
Apr 12 03:32:59 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:59 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:59 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fba3832e0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1'): ({'sde'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Removing single pathed devices sde {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[afa0a3bf-3061-4cac-811d-104e5cfa0d39]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6fbf9b70-191a-4645-b7aa-1bdacfd6134b]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Flushing IO for device /dev/sde {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.013s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0da6bdf8-97f0-4774-9db7-392bc8b4eeea]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-plugged-33b2e7fb-520f-46d4-be86-e492aff77d4b {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-plugged-33b2e7fb-520f-46d4-be86-e492aff77d4b {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:59 user nova-compute[72081]: WARNING nova.compute.manager [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received unexpected event network-vif-plugged-33b2e7fb-520f-46d4-be86-e492aff77d4b for instance with vm_state active and task_state deleting.
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-unplugged-7740f806-8319-484b-8b0f-bffd8c5fe532 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-unplugged-7740f806-8319-484b-8b0f-bffd8c5fe532 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-unplugged-7740f806-8319-484b-8b0f-bffd8c5fe532 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-plugged-7740f806-8319-484b-8b0f-bffd8c5fe532 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-plugged-7740f806-8319-484b-8b0f-bffd8c5fe532 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:59 user nova-compute[72081]: WARNING nova.compute.manager [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received unexpected event network-vif-plugged-7740f806-8319-484b-8b0f-bffd8c5fe532 for instance with vm_state active and task_state deleting.
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-unplugged-91177380-5d77-4bed-9ab0-e9085fcb3d5a {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-unplugged-91177380-5d77-4bed-9ab0-e9085fcb3d5a {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-unplugged-91177380-5d77-4bed-9ab0-e9085fcb3d5a for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-plugged-91177380-5d77-4bed-9ab0-e9085fcb3d5a {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-plugged-91177380-5d77-4bed-9ab0-e9085fcb3d5a {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:59 user nova-compute[72081]: WARNING nova.compute.manager [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received unexpected event network-vif-plugged-91177380-5d77-4bed-9ab0-e9085fcb3d5a for instance with vm_state active and task_state deleting.
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-unplugged-6bb0c89d-0a60-4413-b09d-08d4d6ea3a52 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-unplugged-6bb0c89d-0a60-4413-b09d-08d4d6ea3a52 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-cc2614f4-e08e-47ae-8e9a-b3905bdf61ad req-7d58d6ea-4239-407a-9fa6-d0fc531d3435 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-unplugged-6bb0c89d-0a60-4413-b09d-08d4d6ea3a52 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.057s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[69744331-498e-42af-9a59-f7d78eb5c0cc]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] SCSI volumes sde have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.028s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8c466a59-6452-438c-8037-f261b0fb18fc]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 -p 172.16.0.220:3260 --logout" returned: 0 in 0.083s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8077573b-fad7-46fe-b8ce-cae00ab78be9]: (4, ('Logging out of session [sid: 5, target: iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1, portal: 172.16.0.220,3260]\nLogout of [sid: 5, target: iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 5, target: iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1, portal: 172.16.0.220,3260]
Apr 12 03:32:59 user nova-compute[72081]: Logout of [sid: 5, target: iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1, portal: 172.16.0.220,3260] successful.
Apr 12 03:32:59 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-d4891669-d6eb-4368-a1d1-7840d27e2e0c req-5079b151-59b8-4912-bc7f-b648fa1b5490 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-plugged-be3de447-6d22-4fec-b962-0ebe772ed86f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d4891669-d6eb-4368-a1d1-7840d27e2e0c req-5079b151-59b8-4912-bc7f-b648fa1b5490 service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d4891669-d6eb-4368-a1d1-7840d27e2e0c req-5079b151-59b8-4912-bc7f-b648fa1b5490 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d4891669-d6eb-4368-a1d1-7840d27e2e0c req-5079b151-59b8-4912-bc7f-b648fa1b5490 service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-d4891669-d6eb-4368-a1d1-7840d27e2e0c req-5079b151-59b8-4912-bc7f-b648fa1b5490 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-plugged-be3de447-6d22-4fec-b962-0ebe772ed86f {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:59 user nova-compute[72081]: WARNING nova.compute.manager [req-d4891669-d6eb-4368-a1d1-7840d27e2e0c req-5079b151-59b8-4912-bc7f-b648fa1b5490 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received unexpected event network-vif-plugged-be3de447-6d22-4fec-b962-0ebe772ed86f for instance with vm_state active and task_state deleting.
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a6c8492-c29c-49f6-954b-a9381cd886e1 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.029s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[617c5d33-6aa1-4e69-8b6a-a2a545bb387d]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.392s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] <== disconnect_volume: return (394ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kNcbyUKL', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:32:59 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[974a92f3-3a46-4780-98b8-1007d8e9c14a]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:32:59 user nova-compute[72081]: No targets found.
Apr 12 03:32:59 user nova-compute[72081]: iSNS:
Apr 12 03:32:59 user nova-compute[72081]: No targets found.
Apr 12 03:32:59 user nova-compute[72081]: STATIC:
Apr 12 03:32:59 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93
Apr 12 03:32:59 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:59 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:59 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9
Apr 12 03:32:59 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:59 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:59 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3
Apr 12 03:32:59 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:59 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:59 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3
Apr 12 03:32:59 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:59 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:59 user nova-compute[72081]: FIRMWARE:
Apr 12 03:32:59 user nova-compute[72081]: No targets found.
Apr 12 03:32:59 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:32:59 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:32:59 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a7ba1542-b39d-4c04-b5b0-6e2fd3a3307c]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2da7aa91-48e2-4cb6-9e01-66f35e82546a]: (4, ('tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)\ntcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:59 user nova-compute[72081]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)
Apr 12 03:32:59 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:59 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:59 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsi session list stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:59 user nova-compute[72081]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)
Apr 12 03:32:59 user nova-compute[72081]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 (non-flash)
Apr 12 03:32:59 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:59 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fba3b5120>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9'): ({'sdf'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Removing single pathed devices sdf {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[84c78815-9f3c-4648-bfe2-759aedd5a996]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdf {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdf" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ceff6f08-bcf4-4cef-9616-53e87a8163a9]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Flushing IO for device /dev/sdf {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdf {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdf" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7f8bd084-c37e-4853-b9f8-952ff82dcb7f]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Remove SCSI device /dev/sdf with /sys/block/sdf/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdf/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdf/device/delete" returned: 0 in 0.047s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a148badf-c038-4a0f-888b-c727e23eb7df]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Checking to see if SCSI volumes sdf have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] SCSI volumes sdf have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[323653f4-a439-4f30-8e02-c76d748ba7ca]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 -p 172.16.0.220:3260 --logout" returned: 0 in 0.082s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2cb5b8fa-cdfb-48e8-9e52-578962618bdd]: (4, ('Logging out of session [sid: 6, target: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9, portal: 172.16.0.220,3260]\nLogout of [sid: 6, target: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 6, target: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9, portal: 172.16.0.220,3260]
Apr 12 03:32:59 user nova-compute[72081]: Logout of [sid: 6, target: iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9, portal: 172.16.0.220,3260] successful.
Apr 12 03:32:59 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0aea0a9e-658d-4acc-ba07-3ef1bace5eb9 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.031s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ba035d2b-1f56-41dc-9035-b037933fc960]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.374s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] <== disconnect_volume: return (376ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-be796e52-4ccd-4612-aa2d-6b5b77f260d0 req-b8320e0a-7e80-4629-ab78-4d7d1fcd708f service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-plugged-41626728-b21d-4ef8-9d86-5328d2feb3f9 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-be796e52-4ccd-4612-aa2d-6b5b77f260d0 req-b8320e0a-7e80-4629-ab78-4d7d1fcd708f service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-be796e52-4ccd-4612-aa2d-6b5b77f260d0 req-b8320e0a-7e80-4629-ab78-4d7d1fcd708f service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-be796e52-4ccd-4612-aa2d-6b5b77f260d0 req-b8320e0a-7e80-4629-ab78-4d7d1fcd708f service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-be796e52-4ccd-4612-aa2d-6b5b77f260d0 req-b8320e0a-7e80-4629-ab78-4d7d1fcd708f service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-plugged-41626728-b21d-4ef8-9d86-5328d2feb3f9 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:32:59 user nova-compute[72081]: WARNING nova.compute.manager [req-be796e52-4ccd-4612-aa2d-6b5b77f260d0 req-b8320e0a-7e80-4629-ab78-4d7d1fcd708f service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received unexpected event network-vif-plugged-41626728-b21d-4ef8-9d86-5328d2feb3f9 for instance with vm_state active and task_state deleting.
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-be796e52-4ccd-4612-aa2d-6b5b77f260d0 req-b8320e0a-7e80-4629-ab78-4d7d1fcd708f service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-deleted-8248913f-46c8-4f3d-ab47-0203cb267169 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:59 user nova-compute[72081]: INFO nova.compute.manager [req-be796e52-4ccd-4612-aa2d-6b5b77f260d0 req-b8320e0a-7e80-4629-ab78-4d7d1fcd708f service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Neutron deleted interface 8248913f-46c8-4f3d-ab47-0203cb267169; detaching it from the instance and deleting it from the info cache
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.network.neutron [req-be796e52-4ccd-4612-aa2d-6b5b77f260d0 req-b8320e0a-7e80-4629-ab78-4d7d1fcd708f service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Updating instance_info_cache with network_info: [{"id": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "address": "fa:16:3e:30:84:b9", "network": {"id": "9ea0ab38-8496-4a4b-a54b-fa36e90fd60a", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-849127753-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.24", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapce9193f1-28", "ovs_interfaceid": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "address": "fa:16:3e:56:d1:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.118", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap835b5a2a-4d", "ovs_interfaceid": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "a1aaff41-e246-4887-89c6-a91163d4efd5", "address": "fa:16:3e:d2:e8:92", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.42", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa1aaff41-e2", "ovs_interfaceid": "a1aaff41-e246-4887-89c6-a91163d4efd5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "ac401bfa-a662-4290-af6e-0a39e8fde61c", "address": "fa:16:3e:db:3e:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.161", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapac401bfa-a6", "ovs_interfaceid": "ac401bfa-a662-4290-af6e-0a39e8fde61c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "4e57de22-e876-4bfe-bd07-a1003afd2295", "address": "fa:16:3e:93:42:94", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4e57de22-e8", "ovs_interfaceid": "4e57de22-e876-4bfe-bd07-a1003afd2295", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "21860860-2a08-43dc-bda8-0a1d0fab0634", "address": "fa:16:3e:eb:f1:cc", "network": {"id": "435f9638-5e8f-40a8-b271-a183982c70d5", "bridge": "br-int", "label": "tempest-device-tagging-net2-1738895025", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap21860860-2a", "ovs_interfaceid": "21860860-2a08-43dc-bda8-0a1d0fab0634", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'EUKqgyOv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdg': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:32:59 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f728398d-524b-4f22-b91b-a431d62a952c/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b2ebafdd-faaa-420c-bc25-b32c5d73117b]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:32:59 user nova-compute[72081]: No targets found.
Apr 12 03:32:59 user nova-compute[72081]: iSNS:
Apr 12 03:32:59 user nova-compute[72081]: No targets found.
Apr 12 03:32:59 user nova-compute[72081]: STATIC:
Apr 12 03:32:59 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93
Apr 12 03:32:59 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:59 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:59 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3
Apr 12 03:32:59 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:59 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:59 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3
Apr 12 03:32:59 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:32:59 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:32:59 user nova-compute[72081]: FIRMWARE:
Apr 12 03:32:59 user nova-compute[72081]: No targets found.
Apr 12 03:32:59 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:32:59 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:32:59 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-be796e52-4ccd-4612-aa2d-6b5b77f260d0 req-b8320e0a-7e80-4629-ab78-4d7d1fcd708f service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Detach interface failed, port_id=8248913f-46c8-4f3d-ab47-0203cb267169, reason: Instance 7119b8b3-291b-4868-820e-d7bd3dd3ada4 could not be found. {{(pid=72081) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-be796e52-4ccd-4612-aa2d-6b5b77f260d0 req-b8320e0a-7e80-4629-ab78-4d7d1fcd708f service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-deleted-ac401bfa-a662-4290-af6e-0a39e8fde61c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:59 user nova-compute[72081]: INFO nova.compute.manager [req-be796e52-4ccd-4612-aa2d-6b5b77f260d0 req-b8320e0a-7e80-4629-ab78-4d7d1fcd708f service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Neutron deleted interface ac401bfa-a662-4290-af6e-0a39e8fde61c; detaching it from the instance and deleting it from the info cache
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.network.neutron [req-be796e52-4ccd-4612-aa2d-6b5b77f260d0 req-b8320e0a-7e80-4629-ab78-4d7d1fcd708f service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Updating instance_info_cache with network_info: [{"id": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "address": "fa:16:3e:30:84:b9", "network": {"id": "9ea0ab38-8496-4a4b-a54b-fa36e90fd60a", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-849127753-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.24", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapce9193f1-28", "ovs_interfaceid": "ce9193f1-28a9-4c0f-95ae-e21f8140de19", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "address": "fa:16:3e:56:d1:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.118", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap835b5a2a-4d", "ovs_interfaceid": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "a1aaff41-e246-4887-89c6-a91163d4efd5", "address": "fa:16:3e:d2:e8:92", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.42", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa1aaff41-e2", "ovs_interfaceid": "a1aaff41-e246-4887-89c6-a91163d4efd5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "4e57de22-e876-4bfe-bd07-a1003afd2295", "address": "fa:16:3e:93:42:94", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4e57de22-e8", "ovs_interfaceid": "4e57de22-e876-4bfe-bd07-a1003afd2295", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "21860860-2a08-43dc-bda8-0a1d0fab0634", "address": "fa:16:3e:eb:f1:cc", "network": {"id": "435f9638-5e8f-40a8-b271-a183982c70d5", "bridge": "br-int", "label": "tempest-device-tagging-net2-1738895025", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap21860860-2a", "ovs_interfaceid": "21860860-2a08-43dc-bda8-0a1d0fab0634", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ef23d71e-7f81-4b1e-ad01-9f6e24ad8acc]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-be796e52-4ccd-4612-aa2d-6b5b77f260d0 req-b8320e0a-7e80-4629-ab78-4d7d1fcd708f service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Detach interface failed, port_id=ac401bfa-a662-4290-af6e-0a39e8fde61c, reason: Instance 7119b8b3-291b-4868-820e-d7bd3dd3ada4 could not be found. {{(pid=72081) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-be796e52-4ccd-4612-aa2d-6b5b77f260d0 req-b8320e0a-7e80-4629-ab78-4d7d1fcd708f service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-deleted-ce9193f1-28a9-4c0f-95ae-e21f8140de19 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:32:59 user nova-compute[72081]: INFO nova.compute.manager [req-be796e52-4ccd-4612-aa2d-6b5b77f260d0 req-b8320e0a-7e80-4629-ab78-4d7d1fcd708f service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Neutron deleted interface ce9193f1-28a9-4c0f-95ae-e21f8140de19; detaching it from the instance and deleting it from the info cache
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.network.neutron [req-be796e52-4ccd-4612-aa2d-6b5b77f260d0 req-b8320e0a-7e80-4629-ab78-4d7d1fcd708f service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Updating instance_info_cache with network_info: [{"id": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "address": "fa:16:3e:56:d1:90", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.118", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap835b5a2a-4d", "ovs_interfaceid": "835b5a2a-4d35-4d2a-9fdd-0c78b7def163", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "a1aaff41-e246-4887-89c6-a91163d4efd5", "address": "fa:16:3e:d2:e8:92", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.42", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa1aaff41-e2", "ovs_interfaceid": "a1aaff41-e246-4887-89c6-a91163d4efd5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "4e57de22-e876-4bfe-bd07-a1003afd2295", "address": "fa:16:3e:93:42:94", "network": {"id": "d4fb95ab-11b5-4eff-b746-035d8a799faa", "bridge": "br-int", "label": "tempest-device-tagging-net1-1404366487", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4e57de22-e8", "ovs_interfaceid": "4e57de22-e876-4bfe-bd07-a1003afd2295", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "21860860-2a08-43dc-bda8-0a1d0fab0634", "address": "fa:16:3e:eb:f1:cc", "network": {"id": "435f9638-5e8f-40a8-b271-a183982c70d5", "bridge": "br-int", "label": "tempest-device-tagging-net2-1738895025", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "72d31bafe182459aa8bcb4f1bbb5e6d0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap21860860-2a", "ovs_interfaceid": "21860860-2a08-43dc-bda8-0a1d0fab0634", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b42f5384-6ccf-488d-ba7f-8b929df0a2d9]: (4, ('tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)\ntcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:59 user nova-compute[72081]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)
Apr 12 03:32:59 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:59 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsi session list stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:32:59 user nova-compute[72081]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)
Apr 12 03:32:59 user nova-compute[72081]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 (non-flash)
Apr 12 03:32:59 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fba380f70>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3'): ({'sdg'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Removing single pathed devices sdg {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-be796e52-4ccd-4612-aa2d-6b5b77f260d0 req-b8320e0a-7e80-4629-ab78-4d7d1fcd708f service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Detach interface failed, port_id=ce9193f1-28a9-4c0f-95ae-e21f8140de19, reason: Instance 7119b8b3-291b-4868-820e-d7bd3dd3ada4 could not be found. {{(pid=72081) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[baf40fb6-b61b-456a-b949-71ef9660324b]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:32:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdg {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdg" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0516337e-e347-40dc-a6a8-999efd571f9f]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Flushing IO for device /dev/sdg {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdg {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f728398d-524b-4f22-b91b-a431d62a952c/disk --force-share --output=json" returned: 0 in 0.156s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f728398d-524b-4f22-b91b-a431d62a952c/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdg" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e5de8b21-3d9c-44eb-a279-af696734cf33]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Remove SCSI device /dev/sdg with /sys/block/sdg/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdg/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdg/device/delete" returned: 0 in 0.071s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a66a803a-099d-42d1-b4a1-c8fe125335ec]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Checking to see if SCSI volumes sdg have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] SCSI volumes sdg have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[72c10fe1-f328-4127-a0b1-7abf04451c2d]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f728398d-524b-4f22-b91b-a431d62a952c/disk --force-share --output=json" returned: 0 in 0.150s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Acquiring lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.003s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdf (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdg (vdc) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:33:00 user nova-compute[72081]: INFO nova.compute.manager [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Detaching volume 6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 -p 172.16.0.220:3260 --logout" returned: 0 in 0.062s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[27d64cdc-3da2-467a-94e4-249a093d9d9f]: (4, ('Logging out of session [sid: 7, target: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3, portal: 172.16.0.220,3260]\nLogout of [sid: 7, target: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 7, target: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3, portal: 172.16.0.220,3260]
Apr 12 03:33:00 user nova-compute[72081]: Logout of [sid: 7, target: iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3, portal: 172.16.0.220,3260] successful.
Apr 12 03:33:00 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6f5da98e-cf9d-4e7e-b6fb-30a51a803fc3 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e11cb444-f9a5-4186-93fe-416abeacc4e8]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.442s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] <== disconnect_volume: return (443ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:33:00 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Deleting instance files /opt/stack/data/nova/instances/43289ac5-3300-4e1f-8d07-2d66ee78e7ba_del
Apr 12 03:33:00 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Deletion of /opt/stack/data/nova/instances/43289ac5-3300-4e1f-8d07-2d66ee78e7ba_del complete
Apr 12 03:33:00 user nova-compute[72081]: INFO nova.virt.block_device [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Attempting to driver detach volume 6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 from mountpoint /dev/vdb
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Attempting to detach device vdb from instance ccf9f424-d230-4079-87c5-bb08bc85b0b7 from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:33:00 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:33:00 user nova-compute[72081]:   <source dev="/dev/sdh"/>
Apr 12 03:33:00 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:33:00 user nova-compute[72081]:   <serial>6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3</serial>
Apr 12 03:33:00 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 03:33:00 user nova-compute[72081]: </disk>
Apr 12 03:33:00 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:33:00 user nova-compute[72081]: INFO nova.compute.manager [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Took 3.65 seconds to destroy the instance on the hypervisor.
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:33:00 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Successfully detached device vdb from instance ccf9f424-d230-4079-87c5-bb08bc85b0b7 from the persistent domain config.
Apr 12 03:33:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance ccf9f424-d230-4079-87c5-bb08bc85b0b7 from the live domain config. {{(pid=72081) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:33:00 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:33:00 user nova-compute[72081]:   <source dev="/dev/sdh"/>
Apr 12 03:33:00 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:33:00 user nova-compute[72081]:   <serial>6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3</serial>
Apr 12 03:33:00 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 03:33:00 user nova-compute[72081]: </disk>
Apr 12 03:33:00 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7/disk --force-share --output=json" returned: 0 in 0.158s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdh (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:33:00 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Took 5.05 seconds to deallocate network for instance.
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Received event <DeviceRemovedEvent: 1681270380.9123116, ccf9f424-d230-4079-87c5-bb08bc85b0b7 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72081) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance ccf9f424-d230-4079-87c5-bb08bc85b0b7 {{(pid=72081) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 12 03:33:00 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Successfully detached device vdb from instance ccf9f424-d230-4079-87c5-bb08bc85b0b7 from the live domain config.
Apr 12 03:33:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wPNTeqDg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdh', 'volume_id': '6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:33:00 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:33:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[41d69cd9-7361-4428-b6dc-499680bdffd6]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:33:00 user nova-compute[72081]: No targets found.
Apr 12 03:33:00 user nova-compute[72081]: iSNS:
Apr 12 03:33:00 user nova-compute[72081]: No targets found.
Apr 12 03:33:00 user nova-compute[72081]: STATIC:
Apr 12 03:33:00 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93
Apr 12 03:33:00 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:33:00 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:33:00 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3
Apr 12 03:33:00 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:33:00 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:33:00 user nova-compute[72081]: FIRMWARE:
Apr 12 03:33:00 user nova-compute[72081]: No targets found.
Apr 12 03:33:00 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:33:00 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:33:00 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0d0ad3dc-757f-4936-81e2-2bbabc0fa445]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c08f65ca-9871-4e1a-b60a-660604a28254]: (4, ('tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)\ntcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:33:01 user nova-compute[72081]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)
Apr 12 03:33:01 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] iscsi session list stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 (non-flash)
Apr 12 03:33:01 user nova-compute[72081]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)
Apr 12 03:33:01 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fb4211ea0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3'): ({'sdh'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Removing single pathed devices sdh {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[88b44d6d-5dd8-4479-ba26-edf32a4cf411]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdh {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdh" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5a0a1533-55b6-4350-ba5b-4171215c5ccb]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Flushing IO for device /dev/sdh {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdh {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdh" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[66d247c6-63fe-4b3d-b7e2-f725644810d9]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Remove SCSI device /dev/sdh with /sys/block/sdh/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdh/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdh/device/delete" returned: 0 in 0.046s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c77b0d28-1d73-4d6e-89b3-8329638e6bc1]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Checking to see if SCSI volumes sdh have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] SCSI volumes sdh have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8b09b1d6-1832-40a9-95bf-0cd1683903fe]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --logout" returned: 0 in 0.057s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[aaf28a93-4d22-4ade-a594-2ac101af92d4]: (4, ('Logging out of session [sid: 12, target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3, portal: 172.16.0.220,3260]\nLogout of [sid: 12, target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 12, target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3, portal: 172.16.0.220,3260]
Apr 12 03:33:01 user nova-compute[72081]: Logout of [sid: 12, target: iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3, portal: 172.16.0.220,3260] successful.
Apr 12 03:33:01 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a21ca3f-3f2d-48d7-a96c-c1f123ae3bb3 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c0f76e33-10cc-4a79-9a5d-57ec664fbd91]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.341s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] <== disconnect_volume: return (343ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG nova.compute.manager [req-5f330ef3-46b6-42b5-a64f-d84b7e3a9309 req-420b52ef-3fe8-436a-8ff3-91811be610db service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-plugged-6bb0c89d-0a60-4413-b09d-08d4d6ea3a52 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f330ef3-46b6-42b5-a64f-d84b7e3a9309 req-420b52ef-3fe8-436a-8ff3-91811be610db service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f330ef3-46b6-42b5-a64f-d84b7e3a9309 req-420b52ef-3fe8-436a-8ff3-91811be610db service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f330ef3-46b6-42b5-a64f-d84b7e3a9309 req-420b52ef-3fe8-436a-8ff3-91811be610db service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG nova.compute.manager [req-5f330ef3-46b6-42b5-a64f-d84b7e3a9309 req-420b52ef-3fe8-436a-8ff3-91811be610db service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-plugged-6bb0c89d-0a60-4413-b09d-08d4d6ea3a52 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:33:01 user nova-compute[72081]: WARNING nova.compute.manager [req-5f330ef3-46b6-42b5-a64f-d84b7e3a9309 req-420b52ef-3fe8-436a-8ff3-91811be610db service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received unexpected event network-vif-plugged-6bb0c89d-0a60-4413-b09d-08d4d6ea3a52 for instance with vm_state active and task_state deleting.
Apr 12 03:33:01 user nova-compute[72081]: DEBUG nova.compute.manager [req-5f330ef3-46b6-42b5-a64f-d84b7e3a9309 req-420b52ef-3fe8-436a-8ff3-91811be610db service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-unplugged-e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f330ef3-46b6-42b5-a64f-d84b7e3a9309 req-420b52ef-3fe8-436a-8ff3-91811be610db service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f330ef3-46b6-42b5-a64f-d84b7e3a9309 req-420b52ef-3fe8-436a-8ff3-91811be610db service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f330ef3-46b6-42b5-a64f-d84b7e3a9309 req-420b52ef-3fe8-436a-8ff3-91811be610db service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG nova.compute.manager [req-5f330ef3-46b6-42b5-a64f-d84b7e3a9309 req-420b52ef-3fe8-436a-8ff3-91811be610db service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-unplugged-e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG nova.compute.manager [req-5f330ef3-46b6-42b5-a64f-d84b7e3a9309 req-420b52ef-3fe8-436a-8ff3-91811be610db service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-unplugged-e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG nova.compute.manager [req-5f330ef3-46b6-42b5-a64f-d84b7e3a9309 req-420b52ef-3fe8-436a-8ff3-91811be610db service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-plugged-e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f330ef3-46b6-42b5-a64f-d84b7e3a9309 req-420b52ef-3fe8-436a-8ff3-91811be610db service nova] Acquiring lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f330ef3-46b6-42b5-a64f-d84b7e3a9309 req-420b52ef-3fe8-436a-8ff3-91811be610db service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f330ef3-46b6-42b5-a64f-d84b7e3a9309 req-420b52ef-3fe8-436a-8ff3-91811be610db service nova] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG nova.compute.manager [req-5f330ef3-46b6-42b5-a64f-d84b7e3a9309 req-420b52ef-3fe8-436a-8ff3-91811be610db service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] No waiting events found dispatching network-vif-plugged-e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:33:01 user nova-compute[72081]: WARNING nova.compute.manager [req-5f330ef3-46b6-42b5-a64f-d84b7e3a9309 req-420b52ef-3fe8-436a-8ff3-91811be610db service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received unexpected event network-vif-plugged-e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1 for instance with vm_state active and task_state deleting.
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f1a0f3cf-1004-4ff9-9e41-d03291abbb9a/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f1a0f3cf-1004-4ff9-9e41-d03291abbb9a/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f1a0f3cf-1004-4ff9-9e41-d03291abbb9a/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f1a0f3cf-1004-4ff9-9e41-d03291abbb9a/disk --force-share --output=json" returned: 0 in 0.159s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG nova.compute.manager [req-17d21e05-b6cc-4ff5-a5cc-475496ed6902 req-bff3b4af-7580-414f-9197-f5732ab0796c service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-deleted-4e57de22-e876-4bfe-bd07-a1003afd2295 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:01 user nova-compute[72081]: DEBUG nova.compute.manager [req-17d21e05-b6cc-4ff5-a5cc-475496ed6902 req-bff3b4af-7580-414f-9197-f5732ab0796c service nova] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Received event network-vif-deleted-21860860-2a08-43dc-bda8-0a1d0fab0634 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG nova.objects.instance [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lazy-loading 'flavor' on Instance uuid ccf9f424-d230-4079-87c5-bb08bc85b0b7 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:33:02 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:33:02 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:33:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=7940MB free_disk=26.297012329101562GB free_vcpus=5 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bfcdec32-7a6e-481e-a6eb-9efb4c64fe06 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 2.372s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 56834002-7593-4592-a86d-23bb90d00be0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 2d0904c3-74e0-47bb-a832-758a7035b2f5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 72ba1218-065d-40c4-8503-7c4155e47503 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance ccf9f424-d230-4079-87c5-bb08bc85b0b7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 7119b8b3-291b-4868-820e-d7bd3dd3ada4 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 43289ac5-3300-4e1f-8d07-2d66ee78e7ba actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance f1a0f3cf-1004-4ff9-9e41-d03291abbb9a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance f728398d-524b-4f22-b91b-a431d62a952c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=1664MB phys_disk=40GB used_disk=7GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-eb3a7d24-2385-4205-bc99-ffb6c7e82259 tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Acquiring lock "f1a0f3cf-1004-4ff9-9e41-d03291abbb9a" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-eb3a7d24-2385-4205-bc99-ffb6c7e82259 tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Lock "f1a0f3cf-1004-4ff9-9e41-d03291abbb9a" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-eb3a7d24-2385-4205-bc99-ffb6c7e82259 tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Acquiring lock "f1a0f3cf-1004-4ff9-9e41-d03291abbb9a-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-eb3a7d24-2385-4205-bc99-ffb6c7e82259 tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Lock "f1a0f3cf-1004-4ff9-9e41-d03291abbb9a-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-eb3a7d24-2385-4205-bc99-ffb6c7e82259 tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Lock "f1a0f3cf-1004-4ff9-9e41-d03291abbb9a-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:33:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.420s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:02 user nova-compute[72081]: INFO nova.compute.manager [None req-eb3a7d24-2385-4205-bc99-ffb6c7e82259 tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Terminating instance
Apr 12 03:33:02 user nova-compute[72081]: DEBUG nova.compute.manager [None req-eb3a7d24-2385-4205-bc99-ffb6c7e82259 tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:33:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-e10ac00c-ffa6-45b0-ab05-d1a2901a8c3a req-7309a805-9ac4-429c-8d27-e256b3609ab3 service nova] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Received event network-vif-unplugged-e9d42f8f-a0eb-4058-83e3-f5014b577443 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e10ac00c-ffa6-45b0-ab05-d1a2901a8c3a req-7309a805-9ac4-429c-8d27-e256b3609ab3 service nova] Acquiring lock "f1a0f3cf-1004-4ff9-9e41-d03291abbb9a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e10ac00c-ffa6-45b0-ab05-d1a2901a8c3a req-7309a805-9ac4-429c-8d27-e256b3609ab3 service nova] Lock "f1a0f3cf-1004-4ff9-9e41-d03291abbb9a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e10ac00c-ffa6-45b0-ab05-d1a2901a8c3a req-7309a805-9ac4-429c-8d27-e256b3609ab3 service nova] Lock "f1a0f3cf-1004-4ff9-9e41-d03291abbb9a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-e10ac00c-ffa6-45b0-ab05-d1a2901a8c3a req-7309a805-9ac4-429c-8d27-e256b3609ab3 service nova] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] No waiting events found dispatching network-vif-unplugged-e9d42f8f-a0eb-4058-83e3-f5014b577443 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:33:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-e10ac00c-ffa6-45b0-ab05-d1a2901a8c3a req-7309a805-9ac4-429c-8d27-e256b3609ab3 service nova] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Received event network-vif-unplugged-e9d42f8f-a0eb-4058-83e3-f5014b577443 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:33:03 user nova-compute[72081]: INFO nova.compute.manager [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Took 2.68 seconds to detach 3 volumes for instance.
Apr 12 03:33:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:03 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Instance destroyed successfully.
Apr 12 03:33:03 user nova-compute[72081]: DEBUG nova.objects.instance [None req-eb3a7d24-2385-4205-bc99-ffb6c7e82259 tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Lazy-loading 'resources' on Instance uuid f1a0f3cf-1004-4ff9-9e41-d03291abbb9a {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:33:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-eb3a7d24-2385-4205-bc99-ffb6c7e82259 tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:31:12Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesNegativeTest-instance-1006783758',display_name='tempest-VolumesNegativeTest-instance-1006783758',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesnegativetest-instance-1006783758',id=11,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:31:22Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='f71b2bab30884740b8e6d2c268c7635b',ramdisk_id='',reservation_id='r-ss0xrpn1',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesNegativeTest-833717359',owner_user_name='tempest-VolumesNegativeTest-833717359-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:31:22Z,user_data=None,user_id='1183761babfc4ba3b6f6febe28254c15',uuid=f1a0f3cf-1004-4ff9-9e41-d03291abbb9a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e9d42f8f-a0eb-4058-83e3-f5014b577443", "address": "fa:16:3e:38:4c:25", "network": {"id": "d269b6dc-1cad-4872-9d0f-b4954e3c0aa0", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.139", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "e0a848f7aa0143d98afdd0e4e05fe87c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape9d42f8f-a0", "ovs_interfaceid": "e9d42f8f-a0eb-4058-83e3-f5014b577443", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:33:03 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-eb3a7d24-2385-4205-bc99-ffb6c7e82259 tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Converting VIF {"id": "e9d42f8f-a0eb-4058-83e3-f5014b577443", "address": "fa:16:3e:38:4c:25", "network": {"id": "d269b6dc-1cad-4872-9d0f-b4954e3c0aa0", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.139", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "e0a848f7aa0143d98afdd0e4e05fe87c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape9d42f8f-a0", "ovs_interfaceid": "e9d42f8f-a0eb-4058-83e3-f5014b577443", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:33:03 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-eb3a7d24-2385-4205-bc99-ffb6c7e82259 tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:38:4c:25,bridge_name='br-int',has_traffic_filtering=True,id=e9d42f8f-a0eb-4058-83e3-f5014b577443,network=Network(d269b6dc-1cad-4872-9d0f-b4954e3c0aa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape9d42f8f-a0') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:33:03 user nova-compute[72081]: DEBUG os_vif [None req-eb3a7d24-2385-4205-bc99-ffb6c7e82259 tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:38:4c:25,bridge_name='br-int',has_traffic_filtering=True,id=e9d42f8f-a0eb-4058-83e3-f5014b577443,network=Network(d269b6dc-1cad-4872-9d0f-b4954e3c0aa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape9d42f8f-a0') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:33:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tape9d42f8f-a0, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:33:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:33:03 user nova-compute[72081]: INFO os_vif [None req-eb3a7d24-2385-4205-bc99-ffb6c7e82259 tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:38:4c:25,bridge_name='br-int',has_traffic_filtering=True,id=e9d42f8f-a0eb-4058-83e3-f5014b577443,network=Network(d269b6dc-1cad-4872-9d0f-b4954e3c0aa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape9d42f8f-a0')
Apr 12 03:33:03 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-eb3a7d24-2385-4205-bc99-ffb6c7e82259 tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Deleting instance files /opt/stack/data/nova/instances/f1a0f3cf-1004-4ff9-9e41-d03291abbb9a_del
Apr 12 03:33:03 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-eb3a7d24-2385-4205-bc99-ffb6c7e82259 tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Deletion of /opt/stack/data/nova/instances/f1a0f3cf-1004-4ff9-9e41-d03291abbb9a_del complete
Apr 12 03:33:03 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:33:03 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:33:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-c0c039af-fc2f-4f1a-a10c-885e47fc860f req-47d3e1c8-b588-4343-90ff-cef6ffa7f7b7 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-deleted-33b2e7fb-520f-46d4-be86-e492aff77d4b {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:03 user nova-compute[72081]: INFO nova.compute.manager [req-c0c039af-fc2f-4f1a-a10c-885e47fc860f req-47d3e1c8-b588-4343-90ff-cef6ffa7f7b7 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Neutron deleted interface 33b2e7fb-520f-46d4-be86-e492aff77d4b; detaching it from the instance and deleting it from the info cache
Apr 12 03:33:03 user nova-compute[72081]: DEBUG nova.network.neutron [req-c0c039af-fc2f-4f1a-a10c-885e47fc860f req-47d3e1c8-b588-4343-90ff-cef6ffa7f7b7 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Updating instance_info_cache with network_info: [{"id": "7740f806-8319-484b-8b0f-bffd8c5fe532", "address": "fa:16:3e:10:f4:89", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.24", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7740f806-83", "ovs_interfaceid": "7740f806-8319-484b-8b0f-bffd8c5fe532", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "be3de447-6d22-4fec-b962-0ebe772ed86f", "address": "fa:16:3e:15:de:ed", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.143", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe3de447-6d", "ovs_interfaceid": "be3de447-6d22-4fec-b962-0ebe772ed86f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "address": "fa:16:3e:38:3d:5e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.54", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap91177380-5d", "ovs_interfaceid": "91177380-5d77-4bed-9ab0-e9085fcb3d5a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "address": "fa:16:3e:02:8f:9e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.17", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap41626728-b2", "ovs_interfaceid": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "address": "fa:16:3e:f2:df:e5", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bb0c89d-0a", "ovs_interfaceid": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "address": "fa:16:3e:db:9a:17", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16cfa52-5c", "ovs_interfaceid": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:33:03 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:33:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:33:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Skipping network cache update for instance because it has been migrated to another host. {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9836}}
Apr 12 03:33:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.537s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-c0c039af-fc2f-4f1a-a10c-885e47fc860f req-47d3e1c8-b588-4343-90ff-cef6ffa7f7b7 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Detach interface failed, port_id=33b2e7fb-520f-46d4-be86-e492aff77d4b, reason: Instance 43289ac5-3300-4e1f-8d07-2d66ee78e7ba could not be found. {{(pid=72081) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 12 03:33:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-c0c039af-fc2f-4f1a-a10c-885e47fc860f req-47d3e1c8-b588-4343-90ff-cef6ffa7f7b7 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-deleted-91177380-5d77-4bed-9ab0-e9085fcb3d5a {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:04 user nova-compute[72081]: INFO nova.compute.manager [req-c0c039af-fc2f-4f1a-a10c-885e47fc860f req-47d3e1c8-b588-4343-90ff-cef6ffa7f7b7 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Neutron deleted interface 91177380-5d77-4bed-9ab0-e9085fcb3d5a; detaching it from the instance and deleting it from the info cache
Apr 12 03:33:04 user nova-compute[72081]: DEBUG nova.network.neutron [req-c0c039af-fc2f-4f1a-a10c-885e47fc860f req-47d3e1c8-b588-4343-90ff-cef6ffa7f7b7 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Updating instance_info_cache with network_info: [{"id": "7740f806-8319-484b-8b0f-bffd8c5fe532", "address": "fa:16:3e:10:f4:89", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.24", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7740f806-83", "ovs_interfaceid": "7740f806-8319-484b-8b0f-bffd8c5fe532", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "be3de447-6d22-4fec-b962-0ebe772ed86f", "address": "fa:16:3e:15:de:ed", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.143", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe3de447-6d", "ovs_interfaceid": "be3de447-6d22-4fec-b962-0ebe772ed86f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "address": "fa:16:3e:02:8f:9e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.17", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap41626728-b2", "ovs_interfaceid": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "address": "fa:16:3e:f2:df:e5", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bb0c89d-0a", "ovs_interfaceid": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "address": "fa:16:3e:db:9a:17", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16cfa52-5c", "ovs_interfaceid": "e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:33:04 user nova-compute[72081]: INFO nova.compute.manager [None req-eb3a7d24-2385-4205-bc99-ffb6c7e82259 tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Took 1.11 seconds to destroy the instance on the hypervisor.
Apr 12 03:33:04 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-eb3a7d24-2385-4205-bc99-ffb6c7e82259 tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:33:04 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:33:04 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:33:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-c0c039af-fc2f-4f1a-a10c-885e47fc860f req-47d3e1c8-b588-4343-90ff-cef6ffa7f7b7 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Detach interface failed, port_id=91177380-5d77-4bed-9ab0-e9085fcb3d5a, reason: Instance 43289ac5-3300-4e1f-8d07-2d66ee78e7ba could not be found. {{(pid=72081) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 12 03:33:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-c0c039af-fc2f-4f1a-a10c-885e47fc860f req-47d3e1c8-b588-4343-90ff-cef6ffa7f7b7 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-deleted-e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:04 user nova-compute[72081]: INFO nova.compute.manager [req-c0c039af-fc2f-4f1a-a10c-885e47fc860f req-47d3e1c8-b588-4343-90ff-cef6ffa7f7b7 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Neutron deleted interface e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1; detaching it from the instance and deleting it from the info cache
Apr 12 03:33:04 user nova-compute[72081]: DEBUG nova.network.neutron [req-c0c039af-fc2f-4f1a-a10c-885e47fc860f req-47d3e1c8-b588-4343-90ff-cef6ffa7f7b7 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Updating instance_info_cache with network_info: [{"id": "7740f806-8319-484b-8b0f-bffd8c5fe532", "address": "fa:16:3e:10:f4:89", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.24", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7740f806-83", "ovs_interfaceid": "7740f806-8319-484b-8b0f-bffd8c5fe532", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "be3de447-6d22-4fec-b962-0ebe772ed86f", "address": "fa:16:3e:15:de:ed", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.143", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe3de447-6d", "ovs_interfaceid": "be3de447-6d22-4fec-b962-0ebe772ed86f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "address": "fa:16:3e:02:8f:9e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.17", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap41626728-b2", "ovs_interfaceid": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "address": "fa:16:3e:f2:df:e5", "network": {"id": "2767acc3-5e96-41ad-a7f4-fbd5b4acce3a", "bridge": "br-int", "label": "tempest-device-tagging-net2-97714933", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bb0c89d-0a", "ovs_interfaceid": "6bb0c89d-0a60-4413-b09d-08d4d6ea3a52", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:33:04 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Deleted allocations for instance 7119b8b3-291b-4868-820e-d7bd3dd3ada4
Apr 12 03:33:04 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:33:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-c0c039af-fc2f-4f1a-a10c-885e47fc860f req-47d3e1c8-b588-4343-90ff-cef6ffa7f7b7 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Detach interface failed, port_id=e16cfa52-5c5c-4ef1-bc6e-12fb5b7293b1, reason: Instance 43289ac5-3300-4e1f-8d07-2d66ee78e7ba could not be found. {{(pid=72081) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 12 03:33:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-c0c039af-fc2f-4f1a-a10c-885e47fc860f req-47d3e1c8-b588-4343-90ff-cef6ffa7f7b7 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-deleted-6bb0c89d-0a60-4413-b09d-08d4d6ea3a52 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:04 user nova-compute[72081]: INFO nova.compute.manager [req-c0c039af-fc2f-4f1a-a10c-885e47fc860f req-47d3e1c8-b588-4343-90ff-cef6ffa7f7b7 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Neutron deleted interface 6bb0c89d-0a60-4413-b09d-08d4d6ea3a52; detaching it from the instance and deleting it from the info cache
Apr 12 03:33:04 user nova-compute[72081]: DEBUG nova.network.neutron [req-c0c039af-fc2f-4f1a-a10c-885e47fc860f req-47d3e1c8-b588-4343-90ff-cef6ffa7f7b7 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Updating instance_info_cache with network_info: [{"id": "7740f806-8319-484b-8b0f-bffd8c5fe532", "address": "fa:16:3e:10:f4:89", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.24", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7740f806-83", "ovs_interfaceid": "7740f806-8319-484b-8b0f-bffd8c5fe532", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "be3de447-6d22-4fec-b962-0ebe772ed86f", "address": "fa:16:3e:15:de:ed", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.143", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe3de447-6d", "ovs_interfaceid": "be3de447-6d22-4fec-b962-0ebe772ed86f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "address": "fa:16:3e:02:8f:9e", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.17", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap41626728-b2", "ovs_interfaceid": "41626728-b21d-4ef8-9d86-5328d2feb3f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:33:04 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Took 3.74 seconds to deallocate network for instance.
Apr 12 03:33:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:33:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:33:04 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 03:33:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-c0c039af-fc2f-4f1a-a10c-885e47fc860f req-47d3e1c8-b588-4343-90ff-cef6ffa7f7b7 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Detach interface failed, port_id=6bb0c89d-0a60-4413-b09d-08d4d6ea3a52, reason: Instance 43289ac5-3300-4e1f-8d07-2d66ee78e7ba could not be found. {{(pid=72081) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 12 03:33:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-606964b0-7a57-4045-a2bb-bde14673e662 tempest-TaggedBootDevicesTest_v242-294347777 tempest-TaggedBootDevicesTest_v242-294347777-project-member] Lock "7119b8b3-291b-4868-820e-d7bd3dd3ada4" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 13.828s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:05 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:33:05 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Took 1.06 seconds to deallocate network for instance.
Apr 12 03:33:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-eb3a7d24-2385-4205-bc99-ffb6c7e82259 tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-eb3a7d24-2385-4205-bc99-ffb6c7e82259 tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:05 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Updating instance_info_cache with network_info: [{"id": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "address": "fa:16:3e:c0:f6:ee", "network": {"id": "b4464ad0-6022-43e7-bfe4-229e6a3a0600", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1704232678-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.188", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a027d7708e6747fe970c388d6ba66359", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcff588d4-f4", "ovs_interfaceid": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:33:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:33:05 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 03:33:05 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:33:05 user nova-compute[72081]: DEBUG nova.compute.manager [req-2957a348-4d7b-481c-82c6-3e46aa5eaae4 req-4406de4e-6601-4263-9a2f-698aaac1ffdb service nova] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Received event network-vif-plugged-e9d42f8f-a0eb-4058-83e3-f5014b577443 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2957a348-4d7b-481c-82c6-3e46aa5eaae4 req-4406de4e-6601-4263-9a2f-698aaac1ffdb service nova] Acquiring lock "f1a0f3cf-1004-4ff9-9e41-d03291abbb9a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2957a348-4d7b-481c-82c6-3e46aa5eaae4 req-4406de4e-6601-4263-9a2f-698aaac1ffdb service nova] Lock "f1a0f3cf-1004-4ff9-9e41-d03291abbb9a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2957a348-4d7b-481c-82c6-3e46aa5eaae4 req-4406de4e-6601-4263-9a2f-698aaac1ffdb service nova] Lock "f1a0f3cf-1004-4ff9-9e41-d03291abbb9a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:05 user nova-compute[72081]: DEBUG nova.compute.manager [req-2957a348-4d7b-481c-82c6-3e46aa5eaae4 req-4406de4e-6601-4263-9a2f-698aaac1ffdb service nova] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] No waiting events found dispatching network-vif-plugged-e9d42f8f-a0eb-4058-83e3-f5014b577443 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:33:05 user nova-compute[72081]: WARNING nova.compute.manager [req-2957a348-4d7b-481c-82c6-3e46aa5eaae4 req-4406de4e-6601-4263-9a2f-698aaac1ffdb service nova] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Received unexpected event network-vif-plugged-e9d42f8f-a0eb-4058-83e3-f5014b577443 for instance with vm_state deleted and task_state None.
Apr 12 03:33:05 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-eb3a7d24-2385-4205-bc99-ffb6c7e82259 tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:33:05 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-eb3a7d24-2385-4205-bc99-ffb6c7e82259 tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:33:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-eb3a7d24-2385-4205-bc99-ffb6c7e82259 tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.272s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:05 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-eb3a7d24-2385-4205-bc99-ffb6c7e82259 tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Deleted allocations for instance f1a0f3cf-1004-4ff9-9e41-d03291abbb9a
Apr 12 03:33:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-eb3a7d24-2385-4205-bc99-ffb6c7e82259 tempest-VolumesNegativeTest-833717359 tempest-VolumesNegativeTest-833717359-project-member] Lock "f1a0f3cf-1004-4ff9-9e41-d03291abbb9a" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.641s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:05 user nova-compute[72081]: DEBUG nova.compute.manager [req-338f3719-3a28-47e8-b520-0c18e198289f req-448781e1-1270-4aa5-b761-1a96fee425d5 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Received event network-vif-deleted-41626728-b21d-4ef8-9d86-5328d2feb3f9 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:05 user nova-compute[72081]: INFO nova.compute.manager [req-338f3719-3a28-47e8-b520-0c18e198289f req-448781e1-1270-4aa5-b761-1a96fee425d5 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Neutron deleted interface 41626728-b21d-4ef8-9d86-5328d2feb3f9; detaching it from the instance and deleting it from the info cache
Apr 12 03:33:05 user nova-compute[72081]: DEBUG nova.network.neutron [req-338f3719-3a28-47e8-b520-0c18e198289f req-448781e1-1270-4aa5-b761-1a96fee425d5 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Updating instance_info_cache with network_info: [{"id": "7740f806-8319-484b-8b0f-bffd8c5fe532", "address": "fa:16:3e:10:f4:89", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.24", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7740f806-83", "ovs_interfaceid": "7740f806-8319-484b-8b0f-bffd8c5fe532", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "be3de447-6d22-4fec-b962-0ebe772ed86f", "address": "fa:16:3e:15:de:ed", "network": {"id": "6274e1ca-390b-4378-9386-d41818b17cb3", "bridge": "br-int", "label": "tempest-device-tagging-net1-909271134", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.143", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "bd808e7e237f4c7490abaff5db0e3529", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe3de447-6d", "ovs_interfaceid": "be3de447-6d22-4fec-b962-0ebe772ed86f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:33:06 user nova-compute[72081]: DEBUG nova.compute.manager [req-338f3719-3a28-47e8-b520-0c18e198289f req-448781e1-1270-4aa5-b761-1a96fee425d5 service nova] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Detach interface failed, port_id=41626728-b21d-4ef8-9d86-5328d2feb3f9, reason: Instance 43289ac5-3300-4e1f-8d07-2d66ee78e7ba could not be found. {{(pid=72081) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 12 03:33:06 user nova-compute[72081]: DEBUG nova.compute.manager [req-338f3719-3a28-47e8-b520-0c18e198289f req-448781e1-1270-4aa5-b761-1a96fee425d5 service nova] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Received event network-vif-deleted-e9d42f8f-a0eb-4058-83e3-f5014b577443 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:06 user nova-compute[72081]: INFO nova.compute.manager [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Took 2.44 seconds to detach 3 volumes for instance.
Apr 12 03:33:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:06 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:33:06 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:33:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.270s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:06 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Deleted allocations for instance 43289ac5-3300-4e1f-8d07-2d66ee78e7ba
Apr 12 03:33:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f35b64a0-0096-46a0-a92a-34c6d824acf1 tempest-TaggedBootDevicesTest-1194851633 tempest-TaggedBootDevicesTest-1194851633-project-member] Lock "43289ac5-3300-4e1f-8d07-2d66ee78e7ba" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 10.304s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:07 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681270372.1361206, 7119b8b3-291b-4868-820e-d7bd3dd3ada4 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:33:07 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] VM Stopped (Lifecycle Event)
Apr 12 03:33:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-cc0997b8-e5dc-449a-af1d-862d27cddf1a None None] [instance: 7119b8b3-291b-4868-820e-d7bd3dd3ada4] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:33:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:09 user nova-compute[72081]: DEBUG nova.compute.manager [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:33:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:09 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:33:09 user nova-compute[72081]: INFO nova.compute.claims [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Claim successful on node user
Apr 12 03:33:10 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:33:10 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:33:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.341s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:33:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:33:10 user nova-compute[72081]: DEBUG nova.network.neutron [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:33:10 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:33:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:33:10 user nova-compute[72081]: INFO nova.virt.block_device [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Booting with blank volume at /dev/vda
Apr 12 03:33:10 user nova-compute[72081]: DEBUG nova.policy [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd99be569057f432484e4a60b46882bbd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e1fa352a8c144a1aada7ce0d8818429', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:33:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:10 user nova-compute[72081]: INFO nova.compute.manager [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Detaching volume 263426d6-d387-4a67-9013-5a23e8386c93
Apr 12 03:33:10 user nova-compute[72081]: INFO nova.virt.block_device [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Attempting to driver detach volume 263426d6-d387-4a67-9013-5a23e8386c93 from mountpoint /dev/vdb
Apr 12 03:33:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Attempting to detach device vdb from instance cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0 from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 03:33:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:33:10 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:33:10 user nova-compute[72081]:   <source dev="/dev/sdb"/>
Apr 12 03:33:10 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:33:10 user nova-compute[72081]:   <serial>263426d6-d387-4a67-9013-5a23e8386c93</serial>
Apr 12 03:33:10 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 03:33:10 user nova-compute[72081]: </disk>
Apr 12 03:33:10 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:33:10 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Successfully detached device vdb from instance cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0 from the persistent domain config.
Apr 12 03:33:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0 from the live domain config. {{(pid=72081) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 12 03:33:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:33:10 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:33:10 user nova-compute[72081]:   <source dev="/dev/sdb"/>
Apr 12 03:33:10 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:33:10 user nova-compute[72081]:   <serial>263426d6-d387-4a67-9013-5a23e8386c93</serial>
Apr 12 03:33:10 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 03:33:10 user nova-compute[72081]: </disk>
Apr 12 03:33:10 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:33:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Received event <DeviceRemovedEvent: 1681270390.9208755, cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72081) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 12 03:33:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0 {{(pid=72081) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 12 03:33:10 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Successfully detached device vdb from instance cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0 from the live domain config.
Apr 12 03:33:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:33:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'drGIVBHg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:33:10 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:33:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:33:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:33:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:10 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0fdbd3e2-0309-448d-9daf-fbd2bce676e5]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:33:10 user nova-compute[72081]: No targets found.
Apr 12 03:33:10 user nova-compute[72081]: iSNS:
Apr 12 03:33:10 user nova-compute[72081]: No targets found.
Apr 12 03:33:10 user nova-compute[72081]: STATIC:
Apr 12 03:33:10 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93
Apr 12 03:33:10 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:33:10 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:33:10 user nova-compute[72081]: FIRMWARE:
Apr 12 03:33:10 user nova-compute[72081]: No targets found.
Apr 12 03:33:10 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:33:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:33:10 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:33:10 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:33:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:33:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:10 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b721e57-68f8-4cbb-b061-4e31c0296d41]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:10 user nova-compute[72081]: DEBUG nova.network.neutron [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Successfully created port: a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d779ae24-cd17-408e-a461-367853eac4fe]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)
Apr 12 03:33:11 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 (non-flash)
Apr 12 03:33:11 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fac2ccdc0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93'): ({'sdb'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Removing single pathed devices sdb {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[02aa642d-0c8e-419b-b842-2734f812a92e]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3ee5daad-1c1b-48f4-830d-f5dea7d80720]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Flushing IO for device /dev/sdb {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.012s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[673a5492-7dfb-4ea5-ad2f-50cbf025da5b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.043s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c72a28e-f8fa-43fa-9806-39a77a462ac1]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] SCSI volumes sdb have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9dacd511-4d0f-431e-b008-a117c604b9ce]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 -p 172.16.0.220:3260 --logout" returned: 0 in 0.053s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ba2ad1fc-eb63-4571-829a-15234dcad2e3]: (4, ('Logging out of session [sid: 13, target: iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93, portal: 172.16.0.220,3260]\nLogout of [sid: 13, target: iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 13, target: iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93, portal: 172.16.0.220,3260]
Apr 12 03:33:11 user nova-compute[72081]: Logout of [sid: 13, target: iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93, portal: 172.16.0.220,3260] successful.
Apr 12 03:33:11 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[55f8d727-1c1a-4aa2-a4d1-149b3d0ed960]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.280s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] <== disconnect_volume: return (281ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG nova.network.neutron [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Successfully updated port: a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "refresh_cache-bcd267d2-5b2c-4ed2-b634-0210fae94dfd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquired lock "refresh_cache-bcd267d2-5b2c-4ed2-b634-0210fae94dfd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG nova.network.neutron [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG nova.compute.manager [req-30ee4f79-6a71-4b3d-80a5-53f87002dea4 req-f9b535a3-6362-4ca8-9373-5e2e4c059422 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received event network-changed-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG nova.compute.manager [req-30ee4f79-6a71-4b3d-80a5-53f87002dea4 req-f9b535a3-6362-4ca8-9373-5e2e4c059422 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Refreshing instance network info cache due to event network-changed-a4385731-05f2-4eac-a5bb-9a987a3f5472. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-30ee4f79-6a71-4b3d-80a5-53f87002dea4 req-f9b535a3-6362-4ca8-9373-5e2e4c059422 service nova] Acquiring lock "refresh_cache-bcd267d2-5b2c-4ed2-b634-0210fae94dfd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:33:11 user nova-compute[72081]: DEBUG nova.network.neutron [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG nova.network.neutron [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Updating instance_info_cache with network_info: [{"id": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "address": "fa:16:3e:9d:0c:c2", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa4385731-05", "ovs_interfaceid": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG nova.objects.instance [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lazy-loading 'flavor' on Instance uuid cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Releasing lock "refresh_cache-bcd267d2-5b2c-4ed2-b634-0210fae94dfd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG nova.compute.manager [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Instance network_info: |[{"id": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "address": "fa:16:3e:9d:0c:c2", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa4385731-05", "ovs_interfaceid": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-30ee4f79-6a71-4b3d-80a5-53f87002dea4 req-f9b535a3-6362-4ca8-9373-5e2e4c059422 service nova] Acquired lock "refresh_cache-bcd267d2-5b2c-4ed2-b634-0210fae94dfd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG nova.network.neutron [req-30ee4f79-6a71-4b3d-80a5-53f87002dea4 req-f9b535a3-6362-4ca8-9373-5e2e4c059422 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Refreshing network info cache for port a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f1b931c0-7a5f-4ab1-8fd4-effb8e685208 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.756s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 1.640s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:12 user nova-compute[72081]: INFO nova.compute.manager [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Terminating instance
Apr 12 03:33:12 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:12 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Instance destroyed successfully.
Apr 12 03:33:12 user nova-compute[72081]: DEBUG nova.objects.instance [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lazy-loading 'resources' on Instance uuid cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:30:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-1090048845',display_name='tempest-VolumesSnapshotTestJSON-instance-1090048845',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-1090048845',id=8,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:31:07Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='86f7b42b161f41efb234f0ce70210a08',ramdisk_id='',reservation_id='r-yh0xe0j4',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesSnapshotTestJSON-1513599190',owner_user_name='tempest-VolumesSnapshotTestJSON-1513599190-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:31:07Z,user_data=None,user_id='84ea3a8776a944178ba5a04bce4d9b05',uuid=cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "91c0bcc3-17a9-4444-bb3c-715e45188260", "address": "fa:16:3e:a8:9f:cb", "network": {"id": "07d0f97b-06f9-40b3-8794-e53924379d25", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-2076424457-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "86f7b42b161f41efb234f0ce70210a08", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap91c0bcc3-17", "ovs_interfaceid": "91c0bcc3-17a9-4444-bb3c-715e45188260", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Converting VIF {"id": "91c0bcc3-17a9-4444-bb3c-715e45188260", "address": "fa:16:3e:a8:9f:cb", "network": {"id": "07d0f97b-06f9-40b3-8794-e53924379d25", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-2076424457-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "86f7b42b161f41efb234f0ce70210a08", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap91c0bcc3-17", "ovs_interfaceid": "91c0bcc3-17a9-4444-bb3c-715e45188260", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:a8:9f:cb,bridge_name='br-int',has_traffic_filtering=True,id=91c0bcc3-17a9-4444-bb3c-715e45188260,network=Network(07d0f97b-06f9-40b3-8794-e53924379d25),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap91c0bcc3-17') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG os_vif [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:a8:9f:cb,bridge_name='br-int',has_traffic_filtering=True,id=91c0bcc3-17a9-4444-bb3c-715e45188260,network=Network(07d0f97b-06f9-40b3-8794-e53924379d25),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap91c0bcc3-17') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap91c0bcc3-17, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:33:12 user nova-compute[72081]: INFO os_vif [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:a8:9f:cb,bridge_name='br-int',has_traffic_filtering=True,id=91c0bcc3-17a9-4444-bb3c-715e45188260,network=Network(07d0f97b-06f9-40b3-8794-e53924379d25),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap91c0bcc3-17')
Apr 12 03:33:12 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:263426d6-d387-4a67-9013-5a23e8386c93', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'drGIVBHg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:33:12 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:33:12 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 21 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'iscsiadm -m discoverydb -o show -P 1' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[586c43e6-10c1-48cb-9ee9-4c2337de8222]: Unexpected error while running command.
Apr 12 03:33:12 user nova-compute[72081]: Command: iscsiadm -m discoverydb -o show -P 1
Apr 12 03:33:12 user nova-compute[72081]: Exit code: 21
Apr 12 03:33:12 user nova-compute[72081]: Stdout: 'SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nNo targets found.\nFIRMWARE:\nNo targets found.\n'
Apr 12 03:33:12 user nova-compute[72081]: Stderr: '' {{(pid=82083) _process_cmd /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:479}}
Apr 12 03:33:12 user nova-compute[72081]: Traceback (most recent call last):
Apr 12 03:33:12 user nova-compute[72081]:   File "/usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py", line 476, in _process_cmd
Apr 12 03:33:12 user nova-compute[72081]:     ret = func(*f_args, **f_kwargs)
Apr 12 03:33:12 user nova-compute[72081]:   File "/usr/local/lib/python3.10/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Apr 12 03:33:12 user nova-compute[72081]:     return func(*args, **kwargs)
Apr 12 03:33:12 user nova-compute[72081]:   File "/usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Apr 12 03:33:12 user nova-compute[72081]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Apr 12 03:33:12 user nova-compute[72081]:   File "/usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Apr 12 03:33:12 user nova-compute[72081]:     return putils.execute(on_execute=on_execute,
Apr 12 03:33:12 user nova-compute[72081]:   File "/usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py", line 438, in execute
Apr 12 03:33:12 user nova-compute[72081]:     raise ProcessExecutionError(exit_code=_returncode,
Apr 12 03:33:12 user nova-compute[72081]: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
Apr 12 03:33:12 user nova-compute[72081]: Command: iscsiadm -m discoverydb -o show -P 1
Apr 12 03:33:12 user nova-compute[72081]: Exit code: 21
Apr 12 03:33:12 user nova-compute[72081]: Stdout: 'SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nNo targets found.\nFIRMWARE:\nNo targets found.\n'
Apr 12 03:33:12 user nova-compute[72081]: Stderr: ''
Apr 12 03:33:12 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[586c43e6-10c1-48cb-9ee9-4c2337de8222]: (5, 'oslo_concurrency.processutils.ProcessExecutionError', ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nNo targets found.\nFIRMWARE:\nNo targets found.\n', '', 21, 'iscsiadm -m discoverydb -o show -P 1', None)) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:12 user nova-compute[72081]: ERROR os_brick.initiator.connectors.iscsi [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Exception encountered during portal discovery: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
Apr 12 03:33:12 user nova-compute[72081]: Command: iscsiadm -m discoverydb -o show -P 1
Apr 12 03:33:12 user nova-compute[72081]: Exit code: 21
Apr 12 03:33:12 user nova-compute[72081]: Stdout: 'SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nNo targets found.\nFIRMWARE:\nNo targets found.\n'
Apr 12 03:33:12 user nova-compute[72081]: Stderr: ''
Apr 12 03:33:12 user nova-compute[72081]: ERROR os_brick.initiator.connectors.iscsi Traceback (most recent call last):
Apr 12 03:33:12 user nova-compute[72081]: ERROR os_brick.initiator.connectors.iscsi   File "/usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py", line 206, in _get_ips_iqns_luns
Apr 12 03:33:12 user nova-compute[72081]: ERROR os_brick.initiator.connectors.iscsi     ips_iqns_luns = method(connection_properties)
Apr 12 03:33:12 user nova-compute[72081]: ERROR os_brick.initiator.connectors.iscsi   File "/usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py", line 365, in _get_discoverydb_portals
Apr 12 03:33:12 user nova-compute[72081]: ERROR os_brick.initiator.connectors.iscsi     out = self._run_iscsiadm_bare(['-m', 'discoverydb',
Apr 12 03:33:12 user nova-compute[72081]: ERROR os_brick.initiator.connectors.iscsi   File "/usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py", line 1177, in _run_iscsiadm_bare
Apr 12 03:33:12 user nova-compute[72081]: ERROR os_brick.initiator.connectors.iscsi     (out, err) = self._execute('iscsiadm',
Apr 12 03:33:12 user nova-compute[72081]: ERROR os_brick.initiator.connectors.iscsi   File "/usr/local/lib/python3.10/dist-packages/os_brick/executor.py", line 55, in _execute
Apr 12 03:33:12 user nova-compute[72081]: ERROR os_brick.initiator.connectors.iscsi     result = self.__execute(*args, **kwargs)
Apr 12 03:33:12 user nova-compute[72081]: ERROR os_brick.initiator.connectors.iscsi   File "/usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py", line 172, in execute
Apr 12 03:33:12 user nova-compute[72081]: ERROR os_brick.initiator.connectors.iscsi     return execute_root(*cmd, **kwargs)
Apr 12 03:33:12 user nova-compute[72081]: ERROR os_brick.initiator.connectors.iscsi   File "/usr/local/lib/python3.10/dist-packages/oslo_privsep/priv_context.py", line 271, in _wrap
Apr 12 03:33:12 user nova-compute[72081]: ERROR os_brick.initiator.connectors.iscsi     return self.channel.remote_call(name, args, kwargs,
Apr 12 03:33:12 user nova-compute[72081]: ERROR os_brick.initiator.connectors.iscsi   File "/usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py", line 215, in remote_call
Apr 12 03:33:12 user nova-compute[72081]: ERROR os_brick.initiator.connectors.iscsi     raise exc_type(*result[2])
Apr 12 03:33:12 user nova-compute[72081]: ERROR os_brick.initiator.connectors.iscsi oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
Apr 12 03:33:12 user nova-compute[72081]: ERROR os_brick.initiator.connectors.iscsi Command: iscsiadm -m discoverydb -o show -P 1
Apr 12 03:33:12 user nova-compute[72081]: ERROR os_brick.initiator.connectors.iscsi Exit code: 21
Apr 12 03:33:12 user nova-compute[72081]: ERROR os_brick.initiator.connectors.iscsi Stdout: 'SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nNo targets found.\nFIRMWARE:\nNo targets found.\n'
Apr 12 03:33:12 user nova-compute[72081]: ERROR os_brick.initiator.connectors.iscsi Stderr: ''
Apr 12 03:33:12 user nova-compute[72081]: ERROR os_brick.initiator.connectors.iscsi 
Apr 12 03:33:12 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Skipping cleanup Unable to find target portal 172.16.0.220:3260. {{(pid=72081) _cleanup_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:923}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.031s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] <== disconnect_volume: return (32ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:33:12 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Deleting instance files /opt/stack/data/nova/instances/cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0_del
Apr 12 03:33:12 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Deletion of /opt/stack/data/nova/instances/cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0_del complete
Apr 12 03:33:12 user nova-compute[72081]: DEBUG nova.network.neutron [req-30ee4f79-6a71-4b3d-80a5-53f87002dea4 req-f9b535a3-6362-4ca8-9373-5e2e4c059422 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Updated VIF entry in instance network info cache for port a4385731-05f2-4eac-a5bb-9a987a3f5472. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG nova.network.neutron [req-30ee4f79-6a71-4b3d-80a5-53f87002dea4 req-f9b535a3-6362-4ca8-9373-5e2e4c059422 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Updating instance_info_cache with network_info: [{"id": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "address": "fa:16:3e:9d:0c:c2", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa4385731-05", "ovs_interfaceid": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-30ee4f79-6a71-4b3d-80a5-53f87002dea4 req-f9b535a3-6362-4ca8-9373-5e2e4c059422 service nova] Releasing lock "refresh_cache-bcd267d2-5b2c-4ed2-b634-0210fae94dfd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:33:12 user nova-compute[72081]: INFO nova.compute.manager [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Took 0.69 seconds to destroy the instance on the hypervisor.
Apr 12 03:33:12 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:33:12 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:33:13 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:33:13 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Took 0.43 seconds to deallocate network for instance.
Apr 12 03:33:13 user nova-compute[72081]: WARNING nova.volume.cinder [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Attachment 96baaff2-cf6c-4bb9-bf8b-f22cd9f31adc does not exist. Ignoring.: cinderclient.exceptions.NotFound: Volume attachment could not be found with filter: attachment_id = 96baaff2-cf6c-4bb9-bf8b-f22cd9f31adc. (HTTP 404) (Request-ID: req-7d03e560-c0ed-49e5-bd48-1a53b0ad9833)
Apr 12 03:33:13 user nova-compute[72081]: INFO nova.compute.manager [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Took 0.02 seconds to detach 1 volumes for instance.
Apr 12 03:33:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:13 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:33:13 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:33:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.228s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:13 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Deleted allocations for instance cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0
Apr 12 03:33:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f61389b-b0a8-4efc-bc89-bddbee2d27ce tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.543s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:13 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681270378.6940637, 43289ac5-3300-4e1f-8d07-2d66ee78e7ba => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:33:13 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] VM Stopped (Lifecycle Event)
Apr 12 03:33:13 user nova-compute[72081]: DEBUG nova.compute.manager [None req-5ec1f4e7-15d5-4db7-b56b-d7719f43cdbf None None] [instance: 43289ac5-3300-4e1f-8d07-2d66ee78e7ba] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:33:13 user nova-compute[72081]: DEBUG nova.compute.manager [req-e2d7f937-854b-4e1e-9bb5-cd956a00ef1c req-e81671b2-5f59-4431-a138-4fdc94594fba service nova] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Received event network-vif-unplugged-91c0bcc3-17a9-4444-bb3c-715e45188260 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e2d7f937-854b-4e1e-9bb5-cd956a00ef1c req-e81671b2-5f59-4431-a138-4fdc94594fba service nova] Acquiring lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e2d7f937-854b-4e1e-9bb5-cd956a00ef1c req-e81671b2-5f59-4431-a138-4fdc94594fba service nova] Lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e2d7f937-854b-4e1e-9bb5-cd956a00ef1c req-e81671b2-5f59-4431-a138-4fdc94594fba service nova] Lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:13 user nova-compute[72081]: DEBUG nova.compute.manager [req-e2d7f937-854b-4e1e-9bb5-cd956a00ef1c req-e81671b2-5f59-4431-a138-4fdc94594fba service nova] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] No waiting events found dispatching network-vif-unplugged-91c0bcc3-17a9-4444-bb3c-715e45188260 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:33:13 user nova-compute[72081]: WARNING nova.compute.manager [req-e2d7f937-854b-4e1e-9bb5-cd956a00ef1c req-e81671b2-5f59-4431-a138-4fdc94594fba service nova] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Received unexpected event network-vif-unplugged-91c0bcc3-17a9-4444-bb3c-715e45188260 for instance with vm_state deleted and task_state None.
Apr 12 03:33:13 user nova-compute[72081]: DEBUG nova.compute.manager [req-e2d7f937-854b-4e1e-9bb5-cd956a00ef1c req-e81671b2-5f59-4431-a138-4fdc94594fba service nova] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Received event network-vif-plugged-91c0bcc3-17a9-4444-bb3c-715e45188260 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e2d7f937-854b-4e1e-9bb5-cd956a00ef1c req-e81671b2-5f59-4431-a138-4fdc94594fba service nova] Acquiring lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e2d7f937-854b-4e1e-9bb5-cd956a00ef1c req-e81671b2-5f59-4431-a138-4fdc94594fba service nova] Lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e2d7f937-854b-4e1e-9bb5-cd956a00ef1c req-e81671b2-5f59-4431-a138-4fdc94594fba service nova] Lock "cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:13 user nova-compute[72081]: DEBUG nova.compute.manager [req-e2d7f937-854b-4e1e-9bb5-cd956a00ef1c req-e81671b2-5f59-4431-a138-4fdc94594fba service nova] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] No waiting events found dispatching network-vif-plugged-91c0bcc3-17a9-4444-bb3c-715e45188260 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:33:13 user nova-compute[72081]: WARNING nova.compute.manager [req-e2d7f937-854b-4e1e-9bb5-cd956a00ef1c req-e81671b2-5f59-4431-a138-4fdc94594fba service nova] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Received unexpected event network-vif-plugged-91c0bcc3-17a9-4444-bb3c-715e45188260 for instance with vm_state deleted and task_state None.
Apr 12 03:33:13 user nova-compute[72081]: DEBUG nova.compute.manager [req-e2d7f937-854b-4e1e-9bb5-cd956a00ef1c req-e81671b2-5f59-4431-a138-4fdc94594fba service nova] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Received event network-vif-deleted-91c0bcc3-17a9-4444-bb3c-715e45188260 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:15 user nova-compute[72081]: DEBUG os_brick.utils [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:33:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2a1c7ba7-2e5c-4f8a-881b-97c8f14590e4]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a88a4490-e308-4299-b1d4-0df96701c0da]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3467c5de-8ae3-42e4-95f6-fe2ccc22d25a]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f4ade40c-2355-478a-8c31-7183542301b0]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:33:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:33:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:33:15 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:33:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[76a9c5a7-88e2-4584-ab01-e38e0d6fcf01]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:33:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:33:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:33:15 user nova-compute[72081]: DEBUG os_brick.utils [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] <== get_connector_properties: return (126ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:33:15 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Updating existing volume attachment record: f3dac36a-808b-4e7a-84dd-5079aba3a56e {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:33:16 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:18 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681270383.7382596, f1a0f3cf-1004-4ff9-9e41-d03291abbb9a => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:33:18 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] VM Stopped (Lifecycle Event)
Apr 12 03:33:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-eeb08c91-04bd-491d-827c-ae2515bb7bff None None] [instance: f1a0f3cf-1004-4ff9-9e41-d03291abbb9a] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:33:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:19 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:33:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:19 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:33:19 user nova-compute[72081]: INFO nova.compute.claims [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Claim successful on node user
Apr 12 03:33:19 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:33:19 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:33:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.321s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:19 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:33:19 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:33:19 user nova-compute[72081]: DEBUG nova.network.neutron [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:33:19 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:33:19 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:33:19 user nova-compute[72081]: DEBUG nova.policy [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'a7f422f1a05a4a92ba387e424850b360', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '815b65aa5ad9442db0a087c646f96bbd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:33:19 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:33:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:33:19 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Creating image(s)
Apr 12 03:33:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "/opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "/opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "/opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.165s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.156s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk 1073741824" returned: 0 in 0.064s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.226s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.132s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Checking if we can resize image /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Cannot resize image /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG nova.objects.instance [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'migration_context' on Instance uuid 48977eb2-ce43-4577-a1cb-340e35703dbd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Ensure instance console log exists: /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG nova.compute.manager [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:33:20 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Creating image(s)
Apr 12 03:33:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Did not create local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Ensure instance console log exists: /opt/stack/data/nova/instances/bcd267d2-5b2c-4ed2-b634-0210fae94dfd/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Start _get_guest_xml network_info=[{"id": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "address": "fa:16:3e:9d:0c:c2", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa4385731-05", "ovs_interfaceid": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FfmHBZWX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'bcd267d2-5b2c-4ed2-b634-0210fae94dfd', 'attached_at': '', 'detached_at': '', 'volume_id': 'c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2', 'serial': 'c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2'}, 'disk_bus': 'virtio', 'delete_on_termination': False, 'boot_index': 0, 'guest_format': None, 'device_type': 'disk', 'attachment_id': 'f3dac36a-808b-4e7a-84dd-5079aba3a56e', 'volume_type': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:33:20 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:33:20 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:33:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FfmHBZWX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:33:20 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:33:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:33:20 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:33:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:33:20 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:33:20 user nova-compute[72081]: DEBUG nova.network.neutron [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Successfully created port: 18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260" returned: 21 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2177d79-1557-4ed7-9d54-23400667ca82]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:33:20 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[76e1b8e2-1244-4339-9d7c-85cc821725bd]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2] added
Apr 12 03:33:20 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f0f860bd-321f-49a2-a817-e11efe8861de]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c12dda92-cb92-4367-85a9-bf98ac82693a]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v FfmHBZWX {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v FfmHBZWX" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6bb13ca6-5718-47c8-ba4d-b592df513d12]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'FfmHBZWX'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[db6d5c5f-de5d-4d52-b3b9-9c2f319dbb02]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0d31ea7e-7e9d-4ba6-aec4-f5d2c8e91afe]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Apr 12 03:33:20 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Apr 12 03:33:20 user nova-compute[72081]:  {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:33:20 user nova-compute[72081]: WARNING os_brick.initiator.connectors.iscsi [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Apr 12 03:33:20 user nova-compute[72081]: 
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --login" returned: 0 in 0.045s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4b5316ff-0d71-4076-ad07-6993aea8ceb0]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2, portal: 172.16.0.220,3260]
Apr 12 03:33:20 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2, portal: 172.16.0.220,3260] successful.
Apr 12 03:33:20 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ed3ca45a-17e8-4071-9917-28ac7671bb98]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d42859f3-fd4b-4b5d-ab33-45c311e50aaa]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:33:20 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:33:20 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.231s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] HCTL ('33', '-', '-', 0) found on session 14 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bdf5057e-ab49-4831-810a-1e5f5cdeb7b9]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Searching for a device in session 14 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:33:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FfmHBZWX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.network.neutron [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Successfully updated port: 18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquired lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.network.neutron [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.compute.manager [req-b12f432e-10d7-48c6-9724-63b8bf916e71 req-7d53bac5-10fd-4fc9-859c-a30f42960137 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received event network-changed-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.compute.manager [req-b12f432e-10d7-48c6-9724-63b8bf916e71 req-7d53bac5-10fd-4fc9-859c-a30f42960137 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Refreshing instance network info cache due to event network-changed-18f9f68c-eadd-4582-8f16-b88a13496852. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b12f432e-10d7-48c6-9724-63b8bf916e71 req-7d53bac5-10fd-4fc9-859c-a30f42960137 service nova] Acquiring lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.network.neutron [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.network.neutron [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Updating instance_info_cache with network_info: [{"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Releasing lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Instance network_info: |[{"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b12f432e-10d7-48c6-9724-63b8bf916e71 req-7d53bac5-10fd-4fc9-859c-a30f42960137 service nova] Acquired lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.network.neutron [req-b12f432e-10d7-48c6-9724-63b8bf916e71 req-7d53bac5-10fd-4fc9-859c-a30f42960137 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Refreshing network info cache for port 18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Start _get_guest_xml network_info=[{"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:33:21 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:33:21 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:33:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-2050314981',display_name='tempest-ServerActionsTestJSON-server-2050314981',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-2050314981',id=14,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKmrI9wy1CUIwhlKuXvzk0mD7Pt5nZFvI1CsqS0do6Lk76JQOvGl5lvx9rkz7UAM9WGocGXyWSOUOuKTtYZRD2HqKUbTN4VPOoliE1yuu+s0lx7vuZu45Mj7wDb2S8Ksyw==',key_name='tempest-keypair-364276420',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='815b65aa5ad9442db0a087c646f96bbd',ramdisk_id='',reservation_id='r-usz98205',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestJSON-1898059859',owner_user_name='tempest-ServerActionsTestJSON-1898059859-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:33:20Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a7f422f1a05a4a92ba387e424850b360',uuid=48977eb2-ce43-4577-a1cb-340e35703dbd,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converting VIF {"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.objects.instance [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'pci_devices' on Instance uuid 48977eb2-ce43-4577-a1cb-340e35703dbd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.285s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] <== connect_volume: return (1287ms) {'type': 'block', 'scsi_wwn': '23235646631346235', 'path': '/dev/sdb'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23235646631346235', 'path': '/dev/sdb'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:33:21 user nova-compute[72081]:   <uuid>48977eb2-ce43-4577-a1cb-340e35703dbd</uuid>
Apr 12 03:33:21 user nova-compute[72081]:   <name>instance-0000000e</name>
Apr 12 03:33:21 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:33:21 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:33:21 user nova-compute[72081]:   <metadata>
Apr 12 03:33:21 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:33:21 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:33:21 user nova-compute[72081]:       <nova:name>tempest-ServerActionsTestJSON-server-2050314981</nova:name>
Apr 12 03:33:21 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:33:21</nova:creationTime>
Apr 12 03:33:21 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:33:21 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:33:21 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:33:21 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:33:21 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:33:21 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:33:21 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:33:21 user nova-compute[72081]:       <nova:owner>
Apr 12 03:33:21 user nova-compute[72081]:         <nova:user uuid="a7f422f1a05a4a92ba387e424850b360">tempest-ServerActionsTestJSON-1898059859-project-member</nova:user>
Apr 12 03:33:21 user nova-compute[72081]:         <nova:project uuid="815b65aa5ad9442db0a087c646f96bbd">tempest-ServerActionsTestJSON-1898059859</nova:project>
Apr 12 03:33:21 user nova-compute[72081]:       </nova:owner>
Apr 12 03:33:21 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:33:21 user nova-compute[72081]:       <nova:ports>
Apr 12 03:33:21 user nova-compute[72081]:         <nova:port uuid="18f9f68c-eadd-4582-8f16-b88a13496852">
Apr 12 03:33:21 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 12 03:33:21 user nova-compute[72081]:         </nova:port>
Apr 12 03:33:21 user nova-compute[72081]:       </nova:ports>
Apr 12 03:33:21 user nova-compute[72081]:     </nova:instance>
Apr 12 03:33:21 user nova-compute[72081]:   </metadata>
Apr 12 03:33:21 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:33:21 user nova-compute[72081]:     <system>
Apr 12 03:33:21 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:33:21 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:33:21 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:33:21 user nova-compute[72081]:       <entry name="serial">48977eb2-ce43-4577-a1cb-340e35703dbd</entry>
Apr 12 03:33:21 user nova-compute[72081]:       <entry name="uuid">48977eb2-ce43-4577-a1cb-340e35703dbd</entry>
Apr 12 03:33:21 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:33:21 user nova-compute[72081]:     </system>
Apr 12 03:33:21 user nova-compute[72081]:   </sysinfo>
Apr 12 03:33:21 user nova-compute[72081]:   <os>
Apr 12 03:33:21 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:33:21 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:33:21 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:33:21 user nova-compute[72081]:   </os>
Apr 12 03:33:21 user nova-compute[72081]:   <features>
Apr 12 03:33:21 user nova-compute[72081]:     <acpi/>
Apr 12 03:33:21 user nova-compute[72081]:     <apic/>
Apr 12 03:33:21 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:33:21 user nova-compute[72081]:   </features>
Apr 12 03:33:21 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:33:21 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:33:21 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:33:21 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:33:21 user nova-compute[72081]:   </clock>
Apr 12 03:33:21 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:33:21 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:33:21 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:33:21 user nova-compute[72081]:   </cpu>
Apr 12 03:33:21 user nova-compute[72081]:   <devices>
Apr 12 03:33:21 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:33:21 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:33:21 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk"/>
Apr 12 03:33:21 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:33:21 user nova-compute[72081]:     </disk>
Apr 12 03:33:21 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:33:21 user nova-compute[72081]:       <mac address="fa:16:3e:b3:c5:ca"/>
Apr 12 03:33:21 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:33:21 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:33:21 user nova-compute[72081]:       <target dev="tap18f9f68c-ea"/>
Apr 12 03:33:21 user nova-compute[72081]:     </interface>
Apr 12 03:33:21 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:33:21 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/console.log" append="off"/>
Apr 12 03:33:21 user nova-compute[72081]:     </serial>
Apr 12 03:33:21 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:33:21 user nova-compute[72081]:     <video>
Apr 12 03:33:21 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:33:21 user nova-compute[72081]:     </video>
Apr 12 03:33:21 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:33:21 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:33:21 user nova-compute[72081]:     </rng>
Apr 12 03:33:21 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:33:21 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:33:21 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:33:21 user nova-compute[72081]:     </memballoon>
Apr 12 03:33:21 user nova-compute[72081]:   </devices>
Apr 12 03:33:21 user nova-compute[72081]: </domain>
Apr 12 03:33:21 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:33:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-2050314981',display_name='tempest-ServerActionsTestJSON-server-2050314981',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-2050314981',id=14,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKmrI9wy1CUIwhlKuXvzk0mD7Pt5nZFvI1CsqS0do6Lk76JQOvGl5lvx9rkz7UAM9WGocGXyWSOUOuKTtYZRD2HqKUbTN4VPOoliE1yuu+s0lx7vuZu45Mj7wDb2S8Ksyw==',key_name='tempest-keypair-364276420',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='815b65aa5ad9442db0a087c646f96bbd',ramdisk_id='',reservation_id='r-usz98205',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestJSON-1898059859',owner_user_name='tempest-ServerActionsTestJSON-1898059859-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:33:20Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a7f422f1a05a4a92ba387e424850b360',uuid=48977eb2-ce43-4577-a1cb-340e35703dbd,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converting VIF {"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG os_vif [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap18f9f68c-ea, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap18f9f68c-ea, col_values=(('external_ids', {'iface-id': '18f9f68c-eadd-4582-8f16-b88a13496852', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:b3:c5:ca', 'vm-uuid': '48977eb2-ce43-4577-a1cb-340e35703dbd'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:21 user nova-compute[72081]: INFO os_vif [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea')
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:33:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-206381688',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-206381688',id=13,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6e1fa352a8c144a1aada7ce0d8818429',ramdisk_id='',reservation_id='r-p09cbbxf',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-590302458',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:33:10Z,user_data=None,user_id='d99be569057f432484e4a60b46882bbd',uuid=bcd267d2-5b2c-4ed2-b634-0210fae94dfd,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "address": "fa:16:3e:9d:0c:c2", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa4385731-05", "ovs_interfaceid": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converting VIF {"id": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "address": "fa:16:3e:9d:0c:c2", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa4385731-05", "ovs_interfaceid": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:9d:0c:c2,bridge_name='br-int',has_traffic_filtering=True,id=a4385731-05f2-4eac-a5bb-9a987a3f5472,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa4385731-05') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.objects.instance [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lazy-loading 'pci_devices' on Instance uuid bcd267d2-5b2c-4ed2-b634-0210fae94dfd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:33:21 user nova-compute[72081]:   <uuid>bcd267d2-5b2c-4ed2-b634-0210fae94dfd</uuid>
Apr 12 03:33:21 user nova-compute[72081]:   <name>instance-0000000d</name>
Apr 12 03:33:21 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:33:21 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:33:21 user nova-compute[72081]:   <metadata>
Apr 12 03:33:21 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:33:21 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:33:21 user nova-compute[72081]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-206381688</nova:name>
Apr 12 03:33:21 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:33:20</nova:creationTime>
Apr 12 03:33:21 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:33:21 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:33:21 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:33:21 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:33:21 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:33:21 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:33:21 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:33:21 user nova-compute[72081]:       <nova:owner>
Apr 12 03:33:21 user nova-compute[72081]:         <nova:user uuid="d99be569057f432484e4a60b46882bbd">tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member</nova:user>
Apr 12 03:33:21 user nova-compute[72081]:         <nova:project uuid="6e1fa352a8c144a1aada7ce0d8818429">tempest-ServerBootFromVolumeStableRescueTest-590302458</nova:project>
Apr 12 03:33:21 user nova-compute[72081]:       </nova:owner>
Apr 12 03:33:21 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:33:21 user nova-compute[72081]:       <nova:ports>
Apr 12 03:33:21 user nova-compute[72081]:         <nova:port uuid="a4385731-05f2-4eac-a5bb-9a987a3f5472">
Apr 12 03:33:21 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Apr 12 03:33:21 user nova-compute[72081]:         </nova:port>
Apr 12 03:33:21 user nova-compute[72081]:       </nova:ports>
Apr 12 03:33:21 user nova-compute[72081]:     </nova:instance>
Apr 12 03:33:21 user nova-compute[72081]:   </metadata>
Apr 12 03:33:21 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:33:21 user nova-compute[72081]:     <system>
Apr 12 03:33:21 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:33:21 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:33:21 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:33:21 user nova-compute[72081]:       <entry name="serial">bcd267d2-5b2c-4ed2-b634-0210fae94dfd</entry>
Apr 12 03:33:21 user nova-compute[72081]:       <entry name="uuid">bcd267d2-5b2c-4ed2-b634-0210fae94dfd</entry>
Apr 12 03:33:21 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:33:21 user nova-compute[72081]:     </system>
Apr 12 03:33:21 user nova-compute[72081]:   </sysinfo>
Apr 12 03:33:21 user nova-compute[72081]:   <os>
Apr 12 03:33:21 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:33:21 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:33:21 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:33:21 user nova-compute[72081]:   </os>
Apr 12 03:33:21 user nova-compute[72081]:   <features>
Apr 12 03:33:21 user nova-compute[72081]:     <acpi/>
Apr 12 03:33:21 user nova-compute[72081]:     <apic/>
Apr 12 03:33:21 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:33:21 user nova-compute[72081]:   </features>
Apr 12 03:33:21 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:33:21 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:33:21 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:33:21 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:33:21 user nova-compute[72081]:   </clock>
Apr 12 03:33:21 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:33:21 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:33:21 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:33:21 user nova-compute[72081]:   </cpu>
Apr 12 03:33:21 user nova-compute[72081]:   <devices>
Apr 12 03:33:21 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:33:21 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:33:21 user nova-compute[72081]:       <source dev="/dev/sdb"/>
Apr 12 03:33:21 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:33:21 user nova-compute[72081]:       <serial>c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2</serial>
Apr 12 03:33:21 user nova-compute[72081]:     </disk>
Apr 12 03:33:21 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:33:21 user nova-compute[72081]:       <mac address="fa:16:3e:9d:0c:c2"/>
Apr 12 03:33:21 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:33:21 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:33:21 user nova-compute[72081]:       <target dev="tapa4385731-05"/>
Apr 12 03:33:21 user nova-compute[72081]:     </interface>
Apr 12 03:33:21 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:33:21 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/bcd267d2-5b2c-4ed2-b634-0210fae94dfd/console.log" append="off"/>
Apr 12 03:33:21 user nova-compute[72081]:     </serial>
Apr 12 03:33:21 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:33:21 user nova-compute[72081]:     <video>
Apr 12 03:33:21 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:33:21 user nova-compute[72081]:     </video>
Apr 12 03:33:21 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:33:21 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:33:21 user nova-compute[72081]:     </rng>
Apr 12 03:33:21 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:33:21 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:33:21 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:33:21 user nova-compute[72081]:     </memballoon>
Apr 12 03:33:21 user nova-compute[72081]:   </devices>
Apr 12 03:33:21 user nova-compute[72081]: </domain>
Apr 12 03:33:21 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:33:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-206381688',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-206381688',id=13,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6e1fa352a8c144a1aada7ce0d8818429',ramdisk_id='',reservation_id='r-p09cbbxf',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-590302458',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:33:10Z,user_data=None,user_id='d99be569057f432484e4a60b46882bbd',uuid=bcd267d2-5b2c-4ed2-b634-0210fae94dfd,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "address": "fa:16:3e:9d:0c:c2", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa4385731-05", "ovs_interfaceid": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converting VIF {"id": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "address": "fa:16:3e:9d:0c:c2", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa4385731-05", "ovs_interfaceid": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:9d:0c:c2,bridge_name='br-int',has_traffic_filtering=True,id=a4385731-05f2-4eac-a5bb-9a987a3f5472,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa4385731-05') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG os_vif [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:9d:0c:c2,bridge_name='br-int',has_traffic_filtering=True,id=a4385731-05f2-4eac-a5bb-9a987a3f5472,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa4385731-05') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapa4385731-05, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapa4385731-05, col_values=(('external_ids', {'iface-id': 'a4385731-05f2-4eac-a5bb-9a987a3f5472', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:9d:0c:c2', 'vm-uuid': 'bcd267d2-5b2c-4ed2-b634-0210fae94dfd'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:21 user nova-compute[72081]: INFO os_vif [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:9d:0c:c2,bridge_name='br-int',has_traffic_filtering=True,id=a4385731-05f2-4eac-a5bb-9a987a3f5472,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa4385731-05')
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:33:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] No VIF found with MAC fa:16:3e:b3:c5:ca, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:33:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:33:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] No VIF found with MAC fa:16:3e:9d:0c:c2, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:33:22 user nova-compute[72081]: DEBUG nova.network.neutron [req-b12f432e-10d7-48c6-9724-63b8bf916e71 req-7d53bac5-10fd-4fc9-859c-a30f42960137 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Updated VIF entry in instance network info cache for port 18f9f68c-eadd-4582-8f16-b88a13496852. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:33:22 user nova-compute[72081]: DEBUG nova.network.neutron [req-b12f432e-10d7-48c6-9724-63b8bf916e71 req-7d53bac5-10fd-4fc9-859c-a30f42960137 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Updating instance_info_cache with network_info: [{"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:33:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b12f432e-10d7-48c6-9724-63b8bf916e71 req-7d53bac5-10fd-4fc9-859c-a30f42960137 service nova] Releasing lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:33:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:24 user nova-compute[72081]: DEBUG nova.compute.manager [req-07bc84c9-54bd-453c-a59e-b503fa73a6dd req-0776835a-623e-4af8-962d-deaf3b2f684f service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-07bc84c9-54bd-453c-a59e-b503fa73a6dd req-0776835a-623e-4af8-962d-deaf3b2f684f service nova] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-07bc84c9-54bd-453c-a59e-b503fa73a6dd req-0776835a-623e-4af8-962d-deaf3b2f684f service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-07bc84c9-54bd-453c-a59e-b503fa73a6dd req-0776835a-623e-4af8-962d-deaf3b2f684f service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:24 user nova-compute[72081]: DEBUG nova.compute.manager [req-07bc84c9-54bd-453c-a59e-b503fa73a6dd req-0776835a-623e-4af8-962d-deaf3b2f684f service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] No waiting events found dispatching network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:33:24 user nova-compute[72081]: WARNING nova.compute.manager [req-07bc84c9-54bd-453c-a59e-b503fa73a6dd req-0776835a-623e-4af8-962d-deaf3b2f684f service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received unexpected event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 for instance with vm_state building and task_state spawning.
Apr 12 03:33:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:24 user nova-compute[72081]: DEBUG nova.compute.manager [req-7b21611b-2012-45f4-8b5e-b42629b184c1 req-2f190fac-c436-41ec-ab50-6e1d49e214fa service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7b21611b-2012-45f4-8b5e-b42629b184c1 req-2f190fac-c436-41ec-ab50-6e1d49e214fa service nova] Acquiring lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7b21611b-2012-45f4-8b5e-b42629b184c1 req-2f190fac-c436-41ec-ab50-6e1d49e214fa service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7b21611b-2012-45f4-8b5e-b42629b184c1 req-2f190fac-c436-41ec-ab50-6e1d49e214fa service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:24 user nova-compute[72081]: DEBUG nova.compute.manager [req-7b21611b-2012-45f4-8b5e-b42629b184c1 req-2f190fac-c436-41ec-ab50-6e1d49e214fa service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] No waiting events found dispatching network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:33:24 user nova-compute[72081]: WARNING nova.compute.manager [req-7b21611b-2012-45f4-8b5e-b42629b184c1 req-2f190fac-c436-41ec-ab50-6e1d49e214fa service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received unexpected event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 for instance with vm_state building and task_state spawning.
Apr 12 03:33:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-d3f92bbf-0ba5-4b00-85e3-0df5cec92e70 req-ccd309e2-8182-4b68-97aa-cff5916aaa24 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d3f92bbf-0ba5-4b00-85e3-0df5cec92e70 req-ccd309e2-8182-4b68-97aa-cff5916aaa24 service nova] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d3f92bbf-0ba5-4b00-85e3-0df5cec92e70 req-ccd309e2-8182-4b68-97aa-cff5916aaa24 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d3f92bbf-0ba5-4b00-85e3-0df5cec92e70 req-ccd309e2-8182-4b68-97aa-cff5916aaa24 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-d3f92bbf-0ba5-4b00-85e3-0df5cec92e70 req-ccd309e2-8182-4b68-97aa-cff5916aaa24 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] No waiting events found dispatching network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:33:26 user nova-compute[72081]: WARNING nova.compute.manager [req-d3f92bbf-0ba5-4b00-85e3-0df5cec92e70 req-ccd309e2-8182-4b68-97aa-cff5916aaa24 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received unexpected event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 for instance with vm_state building and task_state spawning.
Apr 12 03:33:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-71876623-dd14-4cb6-8556-cd30033c366f req-954c2c5c-a43d-4472-9128-d1fbc36843a1 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-71876623-dd14-4cb6-8556-cd30033c366f req-954c2c5c-a43d-4472-9128-d1fbc36843a1 service nova] Acquiring lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-71876623-dd14-4cb6-8556-cd30033c366f req-954c2c5c-a43d-4472-9128-d1fbc36843a1 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-71876623-dd14-4cb6-8556-cd30033c366f req-954c2c5c-a43d-4472-9128-d1fbc36843a1 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-71876623-dd14-4cb6-8556-cd30033c366f req-954c2c5c-a43d-4472-9128-d1fbc36843a1 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] No waiting events found dispatching network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:33:26 user nova-compute[72081]: WARNING nova.compute.manager [req-71876623-dd14-4cb6-8556-cd30033c366f req-954c2c5c-a43d-4472-9128-d1fbc36843a1 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received unexpected event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 for instance with vm_state building and task_state spawning.
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270407.1648474, bcd267d2-5b2c-4ed2-b634-0210fae94dfd => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:33:27 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] VM Resumed (Lifecycle Event)
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:33:27 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Instance spawned successfully.
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:33:27 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Instance spawned successfully.
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:33:27 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270407.1686509, 48977eb2-ce43-4577-a1cb-340e35703dbd => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:33:27 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] VM Resumed (Lifecycle Event)
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:33:27 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270407.1764278, bcd267d2-5b2c-4ed2-b634-0210fae94dfd => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:33:27 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] VM Started (Lifecycle Event)
Apr 12 03:33:27 user nova-compute[72081]: INFO nova.compute.manager [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Took 6.80 seconds to spawn the instance on the hypervisor.
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:33:27 user nova-compute[72081]: INFO nova.compute.manager [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Took 7.54 seconds to spawn the instance on the hypervisor.
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:33:27 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270407.1764994, 48977eb2-ce43-4577-a1cb-340e35703dbd => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:33:27 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] VM Started (Lifecycle Event)
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:33:27 user nova-compute[72081]: INFO nova.compute.manager [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Took 17.62 seconds to build instance.
Apr 12 03:33:27 user nova-compute[72081]: INFO nova.compute.manager [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Took 8.18 seconds to build instance.
Apr 12 03:33:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a222a645-4969-4047-a1e7-07b9724c7097 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 17.706s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f2c7053e-f685-410c-a30a-7d7b9cbda009 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 8.272s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681270392.6493993, cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:33:27 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] VM Stopped (Lifecycle Event)
Apr 12 03:33:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9d0b482d-219d-4eda-86e2-81817323a137 None None] [instance: cd5ddb7d-1d62-43a5-a6be-452cd5b1d1b0] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:33:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-75ed2f54-4130-4825-b365-c332ad47aa70 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Acquiring lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-75ed2f54-4130-4825-b365-c332ad47aa70 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-75ed2f54-4130-4825-b365-c332ad47aa70 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Acquiring lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-75ed2f54-4130-4825-b365-c332ad47aa70 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-75ed2f54-4130-4825-b365-c332ad47aa70 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:50 user nova-compute[72081]: INFO nova.compute.manager [None req-75ed2f54-4130-4825-b365-c332ad47aa70 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Terminating instance
Apr 12 03:33:50 user nova-compute[72081]: DEBUG nova.compute.manager [None req-75ed2f54-4130-4825-b365-c332ad47aa70 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:33:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-798ab535-edb2-4c89-9681-17974df2211f tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Acquiring lock "f728398d-524b-4f22-b91b-a431d62a952c" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-798ab535-edb2-4c89-9681-17974df2211f tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "f728398d-524b-4f22-b91b-a431d62a952c" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-798ab535-edb2-4c89-9681-17974df2211f tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Acquiring lock "f728398d-524b-4f22-b91b-a431d62a952c-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-798ab535-edb2-4c89-9681-17974df2211f tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "f728398d-524b-4f22-b91b-a431d62a952c-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-798ab535-edb2-4c89-9681-17974df2211f tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "f728398d-524b-4f22-b91b-a431d62a952c-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:50 user nova-compute[72081]: INFO nova.compute.manager [None req-798ab535-edb2-4c89-9681-17974df2211f tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Terminating instance
Apr 12 03:33:50 user nova-compute[72081]: DEBUG nova.compute.manager [None req-798ab535-edb2-4c89-9681-17974df2211f tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:33:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG nova.compute.manager [req-b8695159-1b4c-402e-a54d-75880362b2fa req-9869c277-12a0-4031-821d-5bb28e6c05cc service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Received event network-vif-unplugged-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b8695159-1b4c-402e-a54d-75880362b2fa req-9869c277-12a0-4031-821d-5bb28e6c05cc service nova] Acquiring lock "f728398d-524b-4f22-b91b-a431d62a952c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b8695159-1b4c-402e-a54d-75880362b2fa req-9869c277-12a0-4031-821d-5bb28e6c05cc service nova] Lock "f728398d-524b-4f22-b91b-a431d62a952c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b8695159-1b4c-402e-a54d-75880362b2fa req-9869c277-12a0-4031-821d-5bb28e6c05cc service nova] Lock "f728398d-524b-4f22-b91b-a431d62a952c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG nova.compute.manager [req-b8695159-1b4c-402e-a54d-75880362b2fa req-9869c277-12a0-4031-821d-5bb28e6c05cc service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] No waiting events found dispatching network-vif-unplugged-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG nova.compute.manager [req-b8695159-1b4c-402e-a54d-75880362b2fa req-9869c277-12a0-4031-821d-5bb28e6c05cc service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Received event network-vif-unplugged-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG nova.compute.manager [req-b83d7fdc-4d9e-46eb-995b-31c5dc3af7db req-0ad50562-47b1-49b4-9b9c-027c04788ffc service nova] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Received event network-vif-unplugged-b58865c9-a096-40af-91cf-828e5131ea22 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b83d7fdc-4d9e-46eb-995b-31c5dc3af7db req-0ad50562-47b1-49b4-9b9c-027c04788ffc service nova] Acquiring lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b83d7fdc-4d9e-46eb-995b-31c5dc3af7db req-0ad50562-47b1-49b4-9b9c-027c04788ffc service nova] Lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b83d7fdc-4d9e-46eb-995b-31c5dc3af7db req-0ad50562-47b1-49b4-9b9c-027c04788ffc service nova] Lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG nova.compute.manager [req-b83d7fdc-4d9e-46eb-995b-31c5dc3af7db req-0ad50562-47b1-49b4-9b9c-027c04788ffc service nova] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] No waiting events found dispatching network-vif-unplugged-b58865c9-a096-40af-91cf-828e5131ea22 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG nova.compute.manager [req-b83d7fdc-4d9e-46eb-995b-31c5dc3af7db req-0ad50562-47b1-49b4-9b9c-027c04788ffc service nova] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Received event network-vif-unplugged-b58865c9-a096-40af-91cf-828e5131ea22 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:51 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Instance destroyed successfully.
Apr 12 03:33:51 user nova-compute[72081]: DEBUG nova.objects.instance [None req-75ed2f54-4130-4825-b365-c332ad47aa70 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lazy-loading 'resources' on Instance uuid ccf9f424-d230-4079-87c5-bb08bc85b0b7 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-75ed2f54-4130-4825-b365-c332ad47aa70 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:30:31Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeSwap-server-571161248',display_name='tempest-TestVolumeSwap-server-571161248',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumeswap-server-571161248',id=6,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBL30RSbxjhGUn3xzLsIU4EV60BoTDcVspvaYQIbHfZgYoaBW0uNt/hjufJbiOeYVn8jEh3f2sIF4VJXBBn5+nGUjfajRfTCakLkfoH9ag8vOMyCnoG9vAKKOAoeQq5klzw==',key_name='tempest-keypair-586254963',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:30:42Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='f51faa215a1546fbaa0db25c9773ba6f',ramdisk_id='',reservation_id='r-6re5vdrm',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeSwap-295510619',owner_user_name='tempest-TestVolumeSwap-295510619-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:30:42Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='09fd9f0996bd48628564e6c170c11a17',uuid=ccf9f424-d230-4079-87c5-bb08bc85b0b7,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "b58865c9-a096-40af-91cf-828e5131ea22", "address": "fa:16:3e:73:fe:b6", "network": {"id": "554fd203-5ff6-4282-87b1-d27aa5dba9e3", "bridge": "br-int", "label": "tempest-TestVolumeSwap-218878042-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.177", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f51faa215a1546fbaa0db25c9773ba6f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb58865c9-a0", "ovs_interfaceid": "b58865c9-a096-40af-91cf-828e5131ea22", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-75ed2f54-4130-4825-b365-c332ad47aa70 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Converting VIF {"id": "b58865c9-a096-40af-91cf-828e5131ea22", "address": "fa:16:3e:73:fe:b6", "network": {"id": "554fd203-5ff6-4282-87b1-d27aa5dba9e3", "bridge": "br-int", "label": "tempest-TestVolumeSwap-218878042-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.177", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f51faa215a1546fbaa0db25c9773ba6f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb58865c9-a0", "ovs_interfaceid": "b58865c9-a096-40af-91cf-828e5131ea22", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-75ed2f54-4130-4825-b365-c332ad47aa70 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:73:fe:b6,bridge_name='br-int',has_traffic_filtering=True,id=b58865c9-a096-40af-91cf-828e5131ea22,network=Network(554fd203-5ff6-4282-87b1-d27aa5dba9e3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb58865c9-a0') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG os_vif [None req-75ed2f54-4130-4825-b365-c332ad47aa70 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:73:fe:b6,bridge_name='br-int',has_traffic_filtering=True,id=b58865c9-a096-40af-91cf-828e5131ea22,network=Network(554fd203-5ff6-4282-87b1-d27aa5dba9e3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb58865c9-a0') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapb58865c9-a0, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:33:51 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Instance destroyed successfully.
Apr 12 03:33:51 user nova-compute[72081]: DEBUG nova.objects.instance [None req-798ab535-edb2-4c89-9681-17974df2211f tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lazy-loading 'resources' on Instance uuid f728398d-524b-4f22-b91b-a431d62a952c {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-798ab535-edb2-4c89-9681-17974df2211f tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:32:00Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-1673161590',display_name='tempest-ServersNegativeTestJSON-server-1673161590',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-1673161590',id=12,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:32:08Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='83a7ef3f97fd47849973d4e4482fb5f9',ramdisk_id='',reservation_id='r-yad7o0xs',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServersNegativeTestJSON-864113759',owner_user_name='tempest-ServersNegativeTestJSON-864113759-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:32:08Z,user_data=None,user_id='3da7394c51664a7caacfda6b7dda9066',uuid=f728398d-524b-4f22-b91b-a431d62a952c,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "b0727a17-1b28-4bd2-a4bd-ba6f779fb19b", "address": "fa:16:3e:80:75:77", "network": {"id": "1a483654-b740-4544-84c9-53a41c11fa88", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1141588183-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "83a7ef3f97fd47849973d4e4482fb5f9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb0727a17-1b", "ovs_interfaceid": "b0727a17-1b28-4bd2-a4bd-ba6f779fb19b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-798ab535-edb2-4c89-9681-17974df2211f tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Converting VIF {"id": "b0727a17-1b28-4bd2-a4bd-ba6f779fb19b", "address": "fa:16:3e:80:75:77", "network": {"id": "1a483654-b740-4544-84c9-53a41c11fa88", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1141588183-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "83a7ef3f97fd47849973d4e4482fb5f9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb0727a17-1b", "ovs_interfaceid": "b0727a17-1b28-4bd2-a4bd-ba6f779fb19b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-798ab535-edb2-4c89-9681-17974df2211f tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:80:75:77,bridge_name='br-int',has_traffic_filtering=True,id=b0727a17-1b28-4bd2-a4bd-ba6f779fb19b,network=Network(1a483654-b740-4544-84c9-53a41c11fa88),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb0727a17-1b') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG os_vif [None req-798ab535-edb2-4c89-9681-17974df2211f tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:80:75:77,bridge_name='br-int',has_traffic_filtering=True,id=b0727a17-1b28-4bd2-a4bd-ba6f779fb19b,network=Network(1a483654-b740-4544-84c9-53a41c11fa88),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb0727a17-1b') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapb0727a17-1b, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:51 user nova-compute[72081]: INFO os_vif [None req-75ed2f54-4130-4825-b365-c332ad47aa70 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:73:fe:b6,bridge_name='br-int',has_traffic_filtering=True,id=b58865c9-a096-40af-91cf-828e5131ea22,network=Network(554fd203-5ff6-4282-87b1-d27aa5dba9e3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb58865c9-a0')
Apr 12 03:33:51 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-75ed2f54-4130-4825-b365-c332ad47aa70 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Deleting instance files /opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7_del
Apr 12 03:33:51 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-75ed2f54-4130-4825-b365-c332ad47aa70 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Deletion of /opt/stack/data/nova/instances/ccf9f424-d230-4079-87c5-bb08bc85b0b7_del complete
Apr 12 03:33:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:33:51 user nova-compute[72081]: INFO os_vif [None req-798ab535-edb2-4c89-9681-17974df2211f tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:80:75:77,bridge_name='br-int',has_traffic_filtering=True,id=b0727a17-1b28-4bd2-a4bd-ba6f779fb19b,network=Network(1a483654-b740-4544-84c9-53a41c11fa88),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb0727a17-1b')
Apr 12 03:33:51 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-798ab535-edb2-4c89-9681-17974df2211f tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Deleting instance files /opt/stack/data/nova/instances/f728398d-524b-4f22-b91b-a431d62a952c_del
Apr 12 03:33:51 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-798ab535-edb2-4c89-9681-17974df2211f tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Deletion of /opt/stack/data/nova/instances/f728398d-524b-4f22-b91b-a431d62a952c_del complete
Apr 12 03:33:51 user nova-compute[72081]: INFO nova.compute.manager [None req-75ed2f54-4130-4825-b365-c332ad47aa70 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Took 1.26 seconds to destroy the instance on the hypervisor.
Apr 12 03:33:51 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-75ed2f54-4130-4825-b365-c332ad47aa70 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:33:51 user nova-compute[72081]: INFO nova.compute.manager [None req-798ab535-edb2-4c89-9681-17974df2211f tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Took 1.14 seconds to destroy the instance on the hypervisor.
Apr 12 03:33:51 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-798ab535-edb2-4c89-9681-17974df2211f tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: f728398d-524b-4f22-b91b-a431d62a952c] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-bd3b2598-2731-4b3b-ad5b-f365ee93cd5e req-62c0d44d-d9fe-44ef-93b3-b4909f256c37 service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Received event network-vif-plugged-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bd3b2598-2731-4b3b-ad5b-f365ee93cd5e req-62c0d44d-d9fe-44ef-93b3-b4909f256c37 service nova] Acquiring lock "f728398d-524b-4f22-b91b-a431d62a952c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bd3b2598-2731-4b3b-ad5b-f365ee93cd5e req-62c0d44d-d9fe-44ef-93b3-b4909f256c37 service nova] Lock "f728398d-524b-4f22-b91b-a431d62a952c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bd3b2598-2731-4b3b-ad5b-f365ee93cd5e req-62c0d44d-d9fe-44ef-93b3-b4909f256c37 service nova] Lock "f728398d-524b-4f22-b91b-a431d62a952c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-bd3b2598-2731-4b3b-ad5b-f365ee93cd5e req-62c0d44d-d9fe-44ef-93b3-b4909f256c37 service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] No waiting events found dispatching network-vif-plugged-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:33:53 user nova-compute[72081]: WARNING nova.compute.manager [req-bd3b2598-2731-4b3b-ad5b-f365ee93cd5e req-62c0d44d-d9fe-44ef-93b3-b4909f256c37 service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Received unexpected event network-vif-plugged-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b for instance with vm_state active and task_state deleting.
Apr 12 03:33:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-bd3b2598-2731-4b3b-ad5b-f365ee93cd5e req-62c0d44d-d9fe-44ef-93b3-b4909f256c37 service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Received event network-vif-plugged-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bd3b2598-2731-4b3b-ad5b-f365ee93cd5e req-62c0d44d-d9fe-44ef-93b3-b4909f256c37 service nova] Acquiring lock "f728398d-524b-4f22-b91b-a431d62a952c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bd3b2598-2731-4b3b-ad5b-f365ee93cd5e req-62c0d44d-d9fe-44ef-93b3-b4909f256c37 service nova] Lock "f728398d-524b-4f22-b91b-a431d62a952c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bd3b2598-2731-4b3b-ad5b-f365ee93cd5e req-62c0d44d-d9fe-44ef-93b3-b4909f256c37 service nova] Lock "f728398d-524b-4f22-b91b-a431d62a952c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-bd3b2598-2731-4b3b-ad5b-f365ee93cd5e req-62c0d44d-d9fe-44ef-93b3-b4909f256c37 service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] No waiting events found dispatching network-vif-plugged-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:33:53 user nova-compute[72081]: WARNING nova.compute.manager [req-bd3b2598-2731-4b3b-ad5b-f365ee93cd5e req-62c0d44d-d9fe-44ef-93b3-b4909f256c37 service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Received unexpected event network-vif-plugged-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b for instance with vm_state active and task_state deleting.
Apr 12 03:33:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-bd3b2598-2731-4b3b-ad5b-f365ee93cd5e req-62c0d44d-d9fe-44ef-93b3-b4909f256c37 service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Received event network-vif-plugged-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bd3b2598-2731-4b3b-ad5b-f365ee93cd5e req-62c0d44d-d9fe-44ef-93b3-b4909f256c37 service nova] Acquiring lock "f728398d-524b-4f22-b91b-a431d62a952c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bd3b2598-2731-4b3b-ad5b-f365ee93cd5e req-62c0d44d-d9fe-44ef-93b3-b4909f256c37 service nova] Lock "f728398d-524b-4f22-b91b-a431d62a952c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bd3b2598-2731-4b3b-ad5b-f365ee93cd5e req-62c0d44d-d9fe-44ef-93b3-b4909f256c37 service nova] Lock "f728398d-524b-4f22-b91b-a431d62a952c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-bd3b2598-2731-4b3b-ad5b-f365ee93cd5e req-62c0d44d-d9fe-44ef-93b3-b4909f256c37 service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] No waiting events found dispatching network-vif-plugged-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:33:53 user nova-compute[72081]: WARNING nova.compute.manager [req-bd3b2598-2731-4b3b-ad5b-f365ee93cd5e req-62c0d44d-d9fe-44ef-93b3-b4909f256c37 service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Received unexpected event network-vif-plugged-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b for instance with vm_state active and task_state deleting.
Apr 12 03:33:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-bd3b2598-2731-4b3b-ad5b-f365ee93cd5e req-62c0d44d-d9fe-44ef-93b3-b4909f256c37 service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Received event network-vif-unplugged-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bd3b2598-2731-4b3b-ad5b-f365ee93cd5e req-62c0d44d-d9fe-44ef-93b3-b4909f256c37 service nova] Acquiring lock "f728398d-524b-4f22-b91b-a431d62a952c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bd3b2598-2731-4b3b-ad5b-f365ee93cd5e req-62c0d44d-d9fe-44ef-93b3-b4909f256c37 service nova] Lock "f728398d-524b-4f22-b91b-a431d62a952c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bd3b2598-2731-4b3b-ad5b-f365ee93cd5e req-62c0d44d-d9fe-44ef-93b3-b4909f256c37 service nova] Lock "f728398d-524b-4f22-b91b-a431d62a952c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-bd3b2598-2731-4b3b-ad5b-f365ee93cd5e req-62c0d44d-d9fe-44ef-93b3-b4909f256c37 service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] No waiting events found dispatching network-vif-unplugged-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:33:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-bd3b2598-2731-4b3b-ad5b-f365ee93cd5e req-62c0d44d-d9fe-44ef-93b3-b4909f256c37 service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Received event network-vif-unplugged-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:33:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-e96565c4-426f-440a-b2d7-45133f49c197 req-c0e5b74d-f6d6-4560-8601-0738865434f8 service nova] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Received event network-vif-plugged-b58865c9-a096-40af-91cf-828e5131ea22 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e96565c4-426f-440a-b2d7-45133f49c197 req-c0e5b74d-f6d6-4560-8601-0738865434f8 service nova] Acquiring lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e96565c4-426f-440a-b2d7-45133f49c197 req-c0e5b74d-f6d6-4560-8601-0738865434f8 service nova] Lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e96565c4-426f-440a-b2d7-45133f49c197 req-c0e5b74d-f6d6-4560-8601-0738865434f8 service nova] Lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-e96565c4-426f-440a-b2d7-45133f49c197 req-c0e5b74d-f6d6-4560-8601-0738865434f8 service nova] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] No waiting events found dispatching network-vif-plugged-b58865c9-a096-40af-91cf-828e5131ea22 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:33:53 user nova-compute[72081]: WARNING nova.compute.manager [req-e96565c4-426f-440a-b2d7-45133f49c197 req-c0e5b74d-f6d6-4560-8601-0738865434f8 service nova] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Received unexpected event network-vif-plugged-b58865c9-a096-40af-91cf-828e5131ea22 for instance with vm_state active and task_state deleting.
Apr 12 03:33:53 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:33:53 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Took 1.88 seconds to deallocate network for instance.
Apr 12 03:33:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-798ab535-edb2-4c89-9681-17974df2211f tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-798ab535-edb2-4c89-9681-17974df2211f tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:53 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:33:53 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Took 2.04 seconds to deallocate network for instance.
Apr 12 03:33:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-75ed2f54-4130-4825-b365-c332ad47aa70 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:54 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-798ab535-edb2-4c89-9681-17974df2211f tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:33:54 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-798ab535-edb2-4c89-9681-17974df2211f tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:33:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-798ab535-edb2-4c89-9681-17974df2211f tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.331s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-75ed2f54-4130-4825-b365-c332ad47aa70 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.197s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:54 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-798ab535-edb2-4c89-9681-17974df2211f tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Deleted allocations for instance f728398d-524b-4f22-b91b-a431d62a952c
Apr 12 03:33:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-798ab535-edb2-4c89-9681-17974df2211f tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "f728398d-524b-4f22-b91b-a431d62a952c" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.569s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:54 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-75ed2f54-4130-4825-b365-c332ad47aa70 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:33:54 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-75ed2f54-4130-4825-b365-c332ad47aa70 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:33:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-75ed2f54-4130-4825-b365-c332ad47aa70 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.286s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:54 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-75ed2f54-4130-4825-b365-c332ad47aa70 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Deleted allocations for instance ccf9f424-d230-4079-87c5-bb08bc85b0b7
Apr 12 03:33:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-75ed2f54-4130-4825-b365-c332ad47aa70 tempest-TestVolumeSwap-295510619 tempest-TestVolumeSwap-295510619-project-member] Lock "ccf9f424-d230-4079-87c5-bb08bc85b0b7" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.979s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:55 user nova-compute[72081]: DEBUG nova.compute.manager [req-9bb37532-9df1-4ab7-8233-3f792222e513 req-21747029-a07f-4be0-8ee3-7fe1d4b24ad2 service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Received event network-vif-plugged-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9bb37532-9df1-4ab7-8233-3f792222e513 req-21747029-a07f-4be0-8ee3-7fe1d4b24ad2 service nova] Acquiring lock "f728398d-524b-4f22-b91b-a431d62a952c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9bb37532-9df1-4ab7-8233-3f792222e513 req-21747029-a07f-4be0-8ee3-7fe1d4b24ad2 service nova] Lock "f728398d-524b-4f22-b91b-a431d62a952c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9bb37532-9df1-4ab7-8233-3f792222e513 req-21747029-a07f-4be0-8ee3-7fe1d4b24ad2 service nova] Lock "f728398d-524b-4f22-b91b-a431d62a952c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:33:55 user nova-compute[72081]: DEBUG nova.compute.manager [req-9bb37532-9df1-4ab7-8233-3f792222e513 req-21747029-a07f-4be0-8ee3-7fe1d4b24ad2 service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] No waiting events found dispatching network-vif-plugged-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:33:55 user nova-compute[72081]: WARNING nova.compute.manager [req-9bb37532-9df1-4ab7-8233-3f792222e513 req-21747029-a07f-4be0-8ee3-7fe1d4b24ad2 service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Received unexpected event network-vif-plugged-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b for instance with vm_state deleted and task_state None.
Apr 12 03:33:55 user nova-compute[72081]: DEBUG nova.compute.manager [req-535c5fb8-a869-4cdd-873d-22d16fe023b7 req-3f22d0bf-3555-4141-bdce-5e349c8baa0b service nova] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Received event network-vif-deleted-b0727a17-1b28-4bd2-a4bd-ba6f779fb19b {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:55 user nova-compute[72081]: DEBUG nova.compute.manager [req-535c5fb8-a869-4cdd-873d-22d16fe023b7 req-3f22d0bf-3555-4141-bdce-5e349c8baa0b service nova] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Received event network-vif-deleted-b58865c9-a096-40af-91cf-828e5131ea22 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:33:56 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:33:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:33:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32" acquired by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:33:57 user nova-compute[72081]: INFO nova.compute.manager [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Unshelving
Apr 12 03:33:57 user nova-compute[72081]: INFO nova.virt.block_device [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Booting with volume e8b7fdb9-05e3-4870-b435-4c063881640f at /dev/vdc
Apr 12 03:33:57 user nova-compute[72081]: DEBUG os_brick.utils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:33:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4ca57462-dd3e-4bbb-bdfe-859f30a18dcd]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9e6e410a-930f-4066-aeb0-9d5d70aa53ff]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:33:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bd0aad2f-6bb6-494a-b65a-db558aedb1e9]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0fbd2fc9-2991-4d88-9ae3-c8d07858476e]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:33:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:33:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:33:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:33:57 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:33:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[fc6d1158-5c43-411a-b0fb-cf104e81830c]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:33:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:33:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:33:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:33:57 user nova-compute[72081]: DEBUG os_brick.utils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] <== get_connector_properties: return (136ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:33:57 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Updating existing volume attachment record: 069cd7a5-8d1b-4562-a19e-8e3235f9c6b9 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:33:57 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:33:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:33:58 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:33:59 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:33:59 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:34:00 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Acquiring lock "72ba1218-065d-40c4-8503-7c4155e47503" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "72ba1218-065d-40c4-8503-7c4155e47503" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG nova.objects.instance [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lazy-loading 'flavor' on Instance uuid 72ba1218-065d-40c4-8503-7c4155e47503 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "72ba1218-065d-40c4-8503-7c4155e47503" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.066s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Acquiring lock "72ba1218-065d-40c4-8503-7c4155e47503" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "72ba1218-065d-40c4-8503-7c4155e47503" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:01 user nova-compute[72081]: INFO nova.compute.manager [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Attaching volume d2e08409-6c4e-44ef-842b-29460b0cb939 to /dev/vdb
Apr 12 03:34:01 user nova-compute[72081]: DEBUG os_brick.utils [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f926b308-1a21-4f49-a7d8-2d2ce378a2c1]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1a348053-09cd-4944-a19c-02353f81ead4]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0fcc67f5-6b56-4340-b536-e7eb6753387f]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[38d9e203-16ac-4b09-bfa5-1191566057cb]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:34:01 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ddde9a7a-14ec-4730-b39d-6aa8f083772e]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG os_brick.utils [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] <== get_connector_properties: return (145ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Updating existing volume attachment record: 73131876-1db8-4d30-9361-a770cf229ba1 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.167s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.168s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.164s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.169s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json" returned: 0 in 0.206s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json" returned: 0 in 0.189s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:02 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:34:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.003s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:03 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lazy-loading 'pci_requests' on Instance uuid b23d4bcd-989a-45e1-89a3-6e3d30229a32 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:34:03 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lazy-loading 'numa_topology' on Instance uuid b23d4bcd-989a-45e1-89a3-6e3d30229a32 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:34:03 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:34:03 user nova-compute[72081]: INFO nova.compute.claims [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Claim successful on node user
Apr 12 03:34:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json" returned: 0 in 0.188s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:03 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:34:03 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:34:03 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8228MB free_disk=26.368526458740234GB free_vcpus=7 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:34:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:03 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:34:03 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:34:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.823s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.102s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:03 user nova-compute[72081]: INFO nova.network.neutron [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Updating port 540b7307-8fea-4b50-bab8-626983135a96 with attributes {'binding:host_id': 'user', 'device_owner': 'compute:nova'}
Apr 12 03:34:04 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 56834002-7593-4592-a86d-23bb90d00be0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:34:04 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 2d0904c3-74e0-47bb-a832-758a7035b2f5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:34:04 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 72ba1218-065d-40c4-8503-7c4155e47503 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:34:04 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance bcd267d2-5b2c-4ed2-b634-0210fae94dfd actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:34:04 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 48977eb2-ce43-4577-a1cb-340e35703dbd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:34:04 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Instance with task_state "spawning" is not being actively managed by this compute host but has allocations referencing this compute node (2203ce45-27b6-4606-b33a-0f78fa74c5d6): {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. Skipping heal of allocations during the task state transition. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1705}}
Apr 12 03:34:04 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 5 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:34:04 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=1152MB phys_disk=40GB used_disk=4GB total_vcpus=12 used_vcpus=5 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:34:04 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:34:04 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:34:04 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:34:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.381s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "refresh_cache-b23d4bcd-989a-45e1-89a3-6e3d30229a32" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:34:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquired lock "refresh_cache-b23d4bcd-989a-45e1-89a3-6e3d30229a32" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:34:04 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:34:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-277a4a62-79fa-4c4b-9f64-5e10a452df6b req-2ca2d666-126a-40ff-a8d8-20e3451ad801 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Received event network-changed-540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:34:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-277a4a62-79fa-4c4b-9f64-5e10a452df6b req-2ca2d666-126a-40ff-a8d8-20e3451ad801 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Refreshing instance network info cache due to event network-changed-540b7307-8fea-4b50-bab8-626983135a96. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:34:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-277a4a62-79fa-4c4b-9f64-5e10a452df6b req-2ca2d666-126a-40ff-a8d8-20e3451ad801 service nova] Acquiring lock "refresh_cache-b23d4bcd-989a-45e1-89a3-6e3d30229a32" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:34:05 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:34:05 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:34:05 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Updating instance_info_cache with network_info: [{"id": "540b7307-8fea-4b50-bab8-626983135a96", "address": "fa:16:3e:8d:a9:ab", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.135", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap540b7307-8f", "ovs_interfaceid": "540b7307-8fea-4b50-bab8-626983135a96", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:34:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Releasing lock "refresh_cache-b23d4bcd-989a-45e1-89a3-6e3d30229a32" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:34:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:34:05 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Creating image(s)
Apr 12 03:34:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "/opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "/opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "/opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:05 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lazy-loading 'trusted_certs' on Instance uuid b23d4bcd-989a-45e1-89a3-6e3d30229a32 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:34:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-277a4a62-79fa-4c4b-9f64-5e10a452df6b req-2ca2d666-126a-40ff-a8d8-20e3451ad801 service nova] Acquired lock "refresh_cache-b23d4bcd-989a-45e1-89a3-6e3d30229a32" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:34:05 user nova-compute[72081]: DEBUG nova.network.neutron [req-277a4a62-79fa-4c4b-9f64-5e10a452df6b req-2ca2d666-126a-40ff-a8d8-20e3451ad801 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Refreshing network info cache for port 540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:34:05 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Didn't find any instances for network info cache update. {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 12 03:34:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "3eb9c7fe4cd49992d2a057f19ea3825baa4d1d9e" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "3eb9c7fe4cd49992d2a057f19ea3825baa4d1d9e" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/3eb9c7fe4cd49992d2a057f19ea3825baa4d1d9e.part --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'VLuEwIle', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:34:06 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:34:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:34:06 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:34:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:34:06 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/3eb9c7fe4cd49992d2a057f19ea3825baa4d1d9e.part --force-share --output=json" returned: 0 in 0.165s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG nova.virt.images [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] 1feda9ce-2630-410a-acd7-d4d319edb2af was qcow2, converting to raw {{(pid=72081) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG nova.privsep.utils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72081) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/3eb9c7fe4cd49992d2a057f19ea3825baa4d1d9e.part /opt/stack/data/nova/instances/_base/3eb9c7fe4cd49992d2a057f19ea3825baa4d1d9e.converted {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c6c44ef3-9675-4b65-8da0-dc5ec2c7b065]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:34:06 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8d3fbfc8-6757-40a6-8020-576c6207c1ea]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939] added
Apr 12 03:34:06 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681270431.5751019, ccf9f424-d230-4079-87c5-bb08bc85b0b7 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:34:06 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] VM Stopped (Lifecycle Event)
Apr 12 03:34:06 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681270431.6295903, f728398d-524b-4f22-b91b-a431d62a952c => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:34:06 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: f728398d-524b-4f22-b91b-a431d62a952c] VM Stopped (Lifecycle Event)
Apr 12 03:34:06 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.161s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[772d45d0-9bb9-43f4-9e99-62e1b6109abb]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ed00dd4d-536b-4710-ab68-38d48a1bba1f None None] [instance: f728398d-524b-4f22-b91b-a431d62a952c] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG nova.compute.manager [None req-44d15227-44d2-471d-a91f-23a134571552 None None] [instance: ccf9f424-d230-4079-87c5-bb08bc85b0b7] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.031s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0dac22cb-e490-4169-9c75-aba9ea072734]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v VLuEwIle {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v VLuEwIle" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f4d2023c-73cc-4508-95e7-d0bcf5dfe9ed]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'VLuEwIle'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.031s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[80b8d688-4716-4e33-86b7-e11bb770ce1f]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/3eb9c7fe4cd49992d2a057f19ea3825baa4d1d9e.part /opt/stack/data/nova/instances/_base/3eb9c7fe4cd49992d2a057f19ea3825baa4d1d9e.converted" returned: 0 in 0.397s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/3eb9c7fe4cd49992d2a057f19ea3825baa4d1d9e.converted --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[17cf7656-9629-4d4f-8528-1f21186e5cc3]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:34:06 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:34:06 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:34:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --login" returned: 0 in 0.068s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b6039acf-ddc5-47fd-9c50-72e3937316b3]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939, portal: 172.16.0.220,3260]
Apr 12 03:34:07 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939, portal: 172.16.0.220,3260] successful.
Apr 12 03:34:07 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Acquiring lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[29809e69-428a-4447-8499-835111ffcc56]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG nova.network.neutron [req-277a4a62-79fa-4c4b-9f64-5e10a452df6b req-2ca2d666-126a-40ff-a8d8-20e3451ad801 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Updated VIF entry in instance network info cache for port 540b7307-8fea-4b50-bab8-626983135a96. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG nova.network.neutron [req-277a4a62-79fa-4c4b-9f64-5e10a452df6b req-2ca2d666-126a-40ff-a8d8-20e3451ad801 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Updating instance_info_cache with network_info: [{"id": "540b7307-8fea-4b50-bab8-626983135a96", "address": "fa:16:3e:8d:a9:ab", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.135", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap540b7307-8f", "ovs_interfaceid": "540b7307-8fea-4b50-bab8-626983135a96", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/3eb9c7fe4cd49992d2a057f19ea3825baa4d1d9e.converted --force-share --output=json" returned: 0 in 0.193s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "3eb9c7fe4cd49992d2a057f19ea3825baa4d1d9e" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.727s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/3eb9c7fe4cd49992d2a057f19ea3825baa4d1d9e --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c106e46c-bf35-4348-98c4-f0f7ef4444fc]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:34:07 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:34:07 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:34:07 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:34:07 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.673s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] HCTL ('34', '-', '-', 0) found on session 15 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-277a4a62-79fa-4c4b-9f64-5e10a452df6b req-2ca2d666-126a-40ff-a8d8-20e3451ad801 service nova] Releasing lock "refresh_cache-b23d4bcd-989a-45e1-89a3-6e3d30229a32" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[14855741-f83c-43f7-bfef-ea6198b6d764]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Searching for a device in session 15 and hctl ['34', '*', '*', 0] yield: None {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/3eb9c7fe4cd49992d2a057f19ea3825baa4d1d9e --force-share --output=json" returned: 0 in 0.158s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "3eb9c7fe4cd49992d2a057f19ea3825baa4d1d9e" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "3eb9c7fe4cd49992d2a057f19ea3825baa4d1d9e" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/3eb9c7fe4cd49992d2a057f19ea3825baa4d1d9e --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:34:07 user nova-compute[72081]: INFO nova.compute.claims [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Claim successful on node user
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/3eb9c7fe4cd49992d2a057f19ea3825baa4d1d9e --force-share --output=json" returned: 0 in 0.173s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/3eb9c7fe4cd49992d2a057f19ea3825baa4d1d9e,backing_fmt=raw /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/3eb9c7fe4cd49992d2a057f19ea3825baa4d1d9e,backing_fmt=raw /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk 1073741824" returned: 0 in 0.054s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "3eb9c7fe4cd49992d2a057f19ea3825baa4d1d9e" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.232s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/3eb9c7fe4cd49992d2a057f19ea3825baa4d1d9e --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/3eb9c7fe4cd49992d2a057f19ea3825baa4d1d9e --force-share --output=json" returned: 0 in 0.155s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lazy-loading 'migration_context' on Instance uuid b23d4bcd-989a-45e1-89a3-6e3d30229a32 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:34:07 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Rebasing disk image.
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): qemu-img rebase -b /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df -F raw /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.589s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:34:07 user nova-compute[72081]: DEBUG nova.network.neutron [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:34:07 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:34:08 user nova-compute[72081]: DEBUG nova.compute.manager [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG nova.policy [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '3a7c2b6c06a745de95fae59f8b02c06d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a327d22640aa4a7caeb35fb0ed2dccc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Searching for a device in session 15 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'VLuEwIle', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.712s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] <== connect_volume: return (1715ms) {'type': 'block', 'scsi_wwn': '23735336265623431', 'path': '/dev/sdc'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23735336265623431', 'path': '/dev/sdc'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG nova.compute.manager [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:34:08 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Creating image(s)
Apr 12 03:34:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Acquiring lock "/opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "/opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "/opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.009s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG nova.objects.instance [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lazy-loading 'flavor' on Instance uuid 72ba1218-065d-40c4-8503-7c4155e47503 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] attach device xml: <disk type="block" device="disk">
Apr 12 03:34:08 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:34:08 user nova-compute[72081]:   <source dev="/dev/sdc"/>
Apr 12 03:34:08 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:34:08 user nova-compute[72081]:   <serial>d2e08409-6c4e-44ef-842b-29460b0cb939</serial>
Apr 12 03:34:08 user nova-compute[72081]: </disk>
Apr 12 03:34:08 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.178s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.003s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.162s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk 1073741824" returned: 0 in 0.056s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.227s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.163s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Checking if we can resize image /opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] No VIF found with MAC fa:16:3e:c0:f6:ee, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk --force-share --output=json" returned: 0 in 0.177s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Cannot resize image /opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:34:08 user nova-compute[72081]: DEBUG nova.objects.instance [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lazy-loading 'migration_context' on Instance uuid 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:34:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:34:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Ensure instance console log exists: /opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:34:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-02260126-ac47-4eb4-9708-14540898f637 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "72ba1218-065d-40c4-8503-7c4155e47503" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 7.770s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:09 user nova-compute[72081]: DEBUG nova.network.neutron [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Successfully created port: b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:34:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.network.neutron [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Successfully updated port: b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Acquiring lock "refresh_cache-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Acquired lock "refresh_cache-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.network.neutron [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.compute.manager [req-cdc974d8-4f2d-4477-9afc-ea8452cb7475 req-c7723c8c-b486-434e-a3df-57c1b637a38e service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Received event network-changed-b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.compute.manager [req-cdc974d8-4f2d-4477-9afc-ea8452cb7475 req-c7723c8c-b486-434e-a3df-57c1b637a38e service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Refreshing instance network info cache due to event network-changed-b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cdc974d8-4f2d-4477-9afc-ea8452cb7475 req-c7723c8c-b486-434e-a3df-57c1b637a38e service nova] Acquiring lock "refresh_cache-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.network.neutron [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "qemu-img rebase -b /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df -F raw /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk" returned: 0 in 2.657s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Ensure instance console log exists: /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Start _get_guest_xml network_info=[{"id": "540b7307-8fea-4b50-bab8-626983135a96", "address": "fa:16:3e:8d:a9:ab", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.135", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap540b7307-8f", "ovs_interfaceid": "540b7307-8fea-4b50-bab8-626983135a96", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdc': {'bus': 'virtio', 'dev': 'vdc', 'type': 'disk'}}} image_meta=ImageMeta(checksum='62b1a9ced39e2cfb3dd1ae1e5cf7b64a',container_format='bare',created_at=2023-04-12T03:32:05Z,direct_url=<?>,disk_format='qcow2',id=1feda9ce-2630-410a-acd7-d4d319edb2af,min_disk=1,min_ram=0,name='tempest-AttachVolumeShelveTestJSON-server-1925048093-shelved',owner='738ebb6b798c4974aa46f9bbe3828279',properties=ImageMetaProps,protected=<?>,size=38600704,status='active',tags=<?>,updated_at=2023-04-12T03:32:09Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vdc', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tWwylQdY', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'attached', 'instance': 'b23d4bcd-989a-45e1-89a3-6e3d30229a32', 'attached_at': '', 'detached_at': '', 'volume_id': 'e8b7fdb9-05e3-4870-b435-4c063881640f', 'serial': 'e8b7fdb9-05e3-4870-b435-4c063881640f'}, 'disk_bus': 'virtio', 'delete_on_termination': False, 'boot_index': None, 'guest_format': None, 'device_type': 'disk', 'attachment_id': '069cd7a5-8d1b-4562-a19e-8e3235f9c6b9', 'volume_type': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:34:10 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:34:10 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='62b1a9ced39e2cfb3dd1ae1e5cf7b64a',container_format='bare',created_at=2023-04-12T03:32:05Z,direct_url=<?>,disk_format='qcow2',id=1feda9ce-2630-410a-acd7-d4d319edb2af,min_disk=1,min_ram=0,name='tempest-AttachVolumeShelveTestJSON-server-1925048093-shelved',owner='738ebb6b798c4974aa46f9bbe3828279',properties=ImageMetaProps,protected=<?>,size=38600704,status='active',tags=<?>,updated_at=2023-04-12T03:32:09Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lazy-loading 'vcpu_model' on Instance uuid b23d4bcd-989a-45e1-89a3-6e3d30229a32 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tWwylQdY', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:34:10 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:34:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:34:10 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:34:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.013s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:34:10 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f -p 172.16.0.220:3260" returned: 21 in 0.057s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[cc335593-67b8-4e3d-bbc5-7344184d43fe]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:34:10 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.network.neutron [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Updating instance_info_cache with network_info: [{"id": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "address": "fa:16:3e:46:28:01", "network": {"id": "8d284b3a-85a8-45a3-845f-271f3fe80540", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-354475413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a327d22640aa4a7caeb35fb0ed2dccc8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb0b22dc0-c2", "ovs_interfaceid": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[359bd8de-138f-439a-a6ca-8ca3c93401d9]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f] added
Apr 12 03:34:10 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.097s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0031e98b-7c2d-488b-9de2-1567a94e17c7]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Releasing lock "refresh_cache-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Instance network_info: |[{"id": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "address": "fa:16:3e:46:28:01", "network": {"id": "8d284b3a-85a8-45a3-845f-271f3fe80540", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-354475413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a327d22640aa4a7caeb35fb0ed2dccc8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb0b22dc0-c2", "ovs_interfaceid": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cdc974d8-4f2d-4477-9afc-ea8452cb7475 req-c7723c8c-b486-434e-a3df-57c1b637a38e service nova] Acquired lock "refresh_cache-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.network.neutron [req-cdc974d8-4f2d-4477-9afc-ea8452cb7475 req-c7723c8c-b486-434e-a3df-57c1b637a38e service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Refreshing network info cache for port b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Start _get_guest_xml network_info=[{"id": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "address": "fa:16:3e:46:28:01", "network": {"id": "8d284b3a-85a8-45a3-845f-271f3fe80540", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-354475413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a327d22640aa4a7caeb35fb0ed2dccc8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb0b22dc0-c2", "ovs_interfaceid": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk.config': {'bus': 'ide', 'dev': 'hda', 'type': 'cdrom'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5f392e69-70cb-48d5-afca-626f31070f3c]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f -p 172.16.0.220:3260 --op update -n node.session.auth.username -v tWwylQdY {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:10 user nova-compute[72081]: INFO nova.compute.manager [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Rescuing
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Acquiring lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Acquired lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:34:10 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:34:10 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:34:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-934756275',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-934756275',id=15,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGqZ2JOd0oMveikCRo3egpYXw6F7M9CSnvg8uuSzQFHFmYKOQ1+3ysT+GbPl8KtibugwjCH/8obQl0aNd57dyywpuoimEfD8++mntdrmImk7chBwlJ4Kqnvt9BRQOwzT8Q==',key_name='tempest-keypair-52848813',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a327d22640aa4a7caeb35fb0ed2dccc8',ramdisk_id='',reservation_id='r-autn96ne',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedAttachmentsTest-1144278074',owner_user_name='tempest-TaggedAttachmentsTest-1144278074-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:34:08Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3a7c2b6c06a745de95fae59f8b02c06d',uuid=37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "address": "fa:16:3e:46:28:01", "network": {"id": "8d284b3a-85a8-45a3-845f-271f3fe80540", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-354475413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a327d22640aa4a7caeb35fb0ed2dccc8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb0b22dc0-c2", "ovs_interfaceid": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Converting VIF {"id": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "address": "fa:16:3e:46:28:01", "network": {"id": "8d284b3a-85a8-45a3-845f-271f3fe80540", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-354475413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a327d22640aa4a7caeb35fb0ed2dccc8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb0b22dc0-c2", "ovs_interfaceid": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:46:28:01,bridge_name='br-int',has_traffic_filtering=True,id=b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958,network=Network(8d284b3a-85a8-45a3-845f-271f3fe80540),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb0b22dc0-c2') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.objects.instance [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lazy-loading 'pci_devices' on Instance uuid 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f -p 172.16.0.220:3260 --op update -n node.session.auth.username -v tWwylQdY" returned: 0 in 0.028s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a74412ee-b0fb-4726-af80-1cd5c052ee5a]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'tWwylQdY'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:34:10 user nova-compute[72081]:   <uuid>37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63</uuid>
Apr 12 03:34:10 user nova-compute[72081]:   <name>instance-0000000f</name>
Apr 12 03:34:10 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:34:10 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:34:10 user nova-compute[72081]:   <metadata>
Apr 12 03:34:10 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:34:10 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:34:10 user nova-compute[72081]:       <nova:name>tempest-device-tagging-server-934756275</nova:name>
Apr 12 03:34:10 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:34:10</nova:creationTime>
Apr 12 03:34:10 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:34:10 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:34:10 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:34:10 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:34:10 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:34:10 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:34:10 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:34:10 user nova-compute[72081]:       <nova:owner>
Apr 12 03:34:10 user nova-compute[72081]:         <nova:user uuid="3a7c2b6c06a745de95fae59f8b02c06d">tempest-TaggedAttachmentsTest-1144278074-project-member</nova:user>
Apr 12 03:34:10 user nova-compute[72081]:         <nova:project uuid="a327d22640aa4a7caeb35fb0ed2dccc8">tempest-TaggedAttachmentsTest-1144278074</nova:project>
Apr 12 03:34:10 user nova-compute[72081]:       </nova:owner>
Apr 12 03:34:10 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:34:10 user nova-compute[72081]:       <nova:ports>
Apr 12 03:34:10 user nova-compute[72081]:         <nova:port uuid="b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958">
Apr 12 03:34:10 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Apr 12 03:34:10 user nova-compute[72081]:         </nova:port>
Apr 12 03:34:10 user nova-compute[72081]:       </nova:ports>
Apr 12 03:34:10 user nova-compute[72081]:     </nova:instance>
Apr 12 03:34:10 user nova-compute[72081]:   </metadata>
Apr 12 03:34:10 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:34:10 user nova-compute[72081]:     <system>
Apr 12 03:34:10 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:34:10 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:34:10 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:34:10 user nova-compute[72081]:       <entry name="serial">37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63</entry>
Apr 12 03:34:10 user nova-compute[72081]:       <entry name="uuid">37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63</entry>
Apr 12 03:34:10 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:34:10 user nova-compute[72081]:     </system>
Apr 12 03:34:10 user nova-compute[72081]:   </sysinfo>
Apr 12 03:34:10 user nova-compute[72081]:   <os>
Apr 12 03:34:10 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:34:10 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:34:10 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:34:10 user nova-compute[72081]:   </os>
Apr 12 03:34:10 user nova-compute[72081]:   <features>
Apr 12 03:34:10 user nova-compute[72081]:     <acpi/>
Apr 12 03:34:10 user nova-compute[72081]:     <apic/>
Apr 12 03:34:10 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:34:10 user nova-compute[72081]:   </features>
Apr 12 03:34:10 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:34:10 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:34:10 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:34:10 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:34:10 user nova-compute[72081]:   </clock>
Apr 12 03:34:10 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:34:10 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:34:10 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:34:10 user nova-compute[72081]:   </cpu>
Apr 12 03:34:10 user nova-compute[72081]:   <devices>
Apr 12 03:34:10 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:34:10 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:34:10 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk"/>
Apr 12 03:34:10 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:34:10 user nova-compute[72081]:     </disk>
Apr 12 03:34:10 user nova-compute[72081]:     <disk type="file" device="cdrom">
Apr 12 03:34:10 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none"/>
Apr 12 03:34:10 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk.config"/>
Apr 12 03:34:10 user nova-compute[72081]:       <target dev="hda" bus="ide"/>
Apr 12 03:34:10 user nova-compute[72081]:     </disk>
Apr 12 03:34:10 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:34:10 user nova-compute[72081]:       <mac address="fa:16:3e:46:28:01"/>
Apr 12 03:34:10 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:34:10 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:34:10 user nova-compute[72081]:       <target dev="tapb0b22dc0-c2"/>
Apr 12 03:34:10 user nova-compute[72081]:     </interface>
Apr 12 03:34:10 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:34:10 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/console.log" append="off"/>
Apr 12 03:34:10 user nova-compute[72081]:     </serial>
Apr 12 03:34:10 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:34:10 user nova-compute[72081]:     <video>
Apr 12 03:34:10 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:34:10 user nova-compute[72081]:     </video>
Apr 12 03:34:10 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:34:10 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:34:10 user nova-compute[72081]:     </rng>
Apr 12 03:34:10 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:34:10 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:34:10 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:34:10 user nova-compute[72081]:     </memballoon>
Apr 12 03:34:10 user nova-compute[72081]:   </devices>
Apr 12 03:34:10 user nova-compute[72081]: </domain>
Apr 12 03:34:10 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:34:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-934756275',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-934756275',id=15,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGqZ2JOd0oMveikCRo3egpYXw6F7M9CSnvg8uuSzQFHFmYKOQ1+3ysT+GbPl8KtibugwjCH/8obQl0aNd57dyywpuoimEfD8++mntdrmImk7chBwlJ4Kqnvt9BRQOwzT8Q==',key_name='tempest-keypair-52848813',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a327d22640aa4a7caeb35fb0ed2dccc8',ramdisk_id='',reservation_id='r-autn96ne',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedAttachmentsTest-1144278074',owner_user_name='tempest-TaggedAttachmentsTest-1144278074-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:34:08Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3a7c2b6c06a745de95fae59f8b02c06d',uuid=37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "address": "fa:16:3e:46:28:01", "network": {"id": "8d284b3a-85a8-45a3-845f-271f3fe80540", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-354475413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a327d22640aa4a7caeb35fb0ed2dccc8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb0b22dc0-c2", "ovs_interfaceid": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Converting VIF {"id": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "address": "fa:16:3e:46:28:01", "network": {"id": "8d284b3a-85a8-45a3-845f-271f3fe80540", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-354475413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a327d22640aa4a7caeb35fb0ed2dccc8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb0b22dc0-c2", "ovs_interfaceid": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:46:28:01,bridge_name='br-int',has_traffic_filtering=True,id=b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958,network=Network(8d284b3a-85a8-45a3-845f-271f3fe80540),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb0b22dc0-c2') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG os_vif [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:46:28:01,bridge_name='br-int',has_traffic_filtering=True,id=b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958,network=Network(8d284b3a-85a8-45a3-845f-271f3fe80540),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb0b22dc0-c2') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapb0b22dc0-c2, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapb0b22dc0-c2, col_values=(('external_ids', {'iface-id': 'b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:46:28:01', 'vm-uuid': '37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.031s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c67d2c3f-1e3c-428d-a083-5c6b3d96e07b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:10 user nova-compute[72081]: INFO os_vif [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:46:28:01,bridge_name='br-int',has_traffic_filtering=True,id=b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958,network=Network(8d284b3a-85a8-45a3-845f-271f3fe80540),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb0b22dc0-c2')
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[472f1a82-d30f-428d-9016-e28b1a49d919]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:34:10 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:34:10 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:34:10 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:34:10 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:34:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:34:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] No BDM found with device name hda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:34:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] No VIF found with MAC fa:16:3e:46:28:01, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:34:11 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Using config drive
Apr 12 03:34:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f -p 172.16.0.220:3260 --login" returned: 0 in 0.077s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5e8cde95-641e-4ff2-84e9-3812998cbfe0]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f, portal: 172.16.0.220,3260]
Apr 12 03:34:11 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f, portal: 172.16.0.220,3260] successful.
Apr 12 03:34:11 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:34:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4b1fd391-8112-4aa5-9303-74754545a866]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:34:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[20f455ad-371c-491c-91cb-0d937ffc5eca]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:34:11 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:34:11 user nova-compute[72081]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f (non-flash)
Apr 12 03:34:11 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:34:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:34:11 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:34:11 user nova-compute[72081]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f (non-flash)
Apr 12 03:34:11 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:34:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.548s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:34:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:34:11 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] HCTL ('35', '-', '-', 0) found on session 16 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:34:11 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:34:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c81fd8db-130c-45b8-a2c6-fc7016c73b68]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:11 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Searching for a device in session 16 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:34:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tWwylQdY', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:34:11 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Creating config drive at /opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk.config
Apr 12 03:34:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Running cmd (subprocess): genisoimage -o /opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpxphnvsuq {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] CMD "genisoimage -o /opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpxphnvsuq" returned: 0 in 0.056s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG nova.network.neutron [req-cdc974d8-4f2d-4477-9afc-ea8452cb7475 req-c7723c8c-b486-434e-a3df-57c1b637a38e service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Updated VIF entry in instance network info cache for port b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG nova.network.neutron [req-cdc974d8-4f2d-4477-9afc-ea8452cb7475 req-c7723c8c-b486-434e-a3df-57c1b637a38e service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Updating instance_info_cache with network_info: [{"id": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "address": "fa:16:3e:46:28:01", "network": {"id": "8d284b3a-85a8-45a3-845f-271f3fe80540", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-354475413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a327d22640aa4a7caeb35fb0ed2dccc8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb0b22dc0-c2", "ovs_interfaceid": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cdc974d8-4f2d-4477-9afc-ea8452cb7475 req-c7723c8c-b486-434e-a3df-57c1b637a38e service nova] Releasing lock "refresh_cache-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.596s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] <== connect_volume: return (1599ms) {'type': 'block', 'scsi_wwn': '23635366537633463', 'path': '/dev/sdd'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23635366537633463', 'path': '/dev/sdd'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-12T03:30:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-1925048093',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-1925048093',id=3,image_ref='1feda9ce-2630-410a-acd7-d4d319edb2af',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name='tempest-keypair-892402082',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:30:30Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='738ebb6b798c4974aa46f9bbe3828279',ramdisk_id='',reservation_id='r-ybseqfps',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-2139975165',owner_user_name='tempest-AttachVolumeShelveTestJSON-2139975165-project-member',shelved_at='2023-04-12T03:32:09.253612',shelved_host='user',shelved_image_id='1feda9ce-2630-410a-acd7-d4d319edb2af'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:33:57Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='de1fe5db01624224ab7b615c908d990b',uuid=b23d4bcd-989a-45e1-89a3-6e3d30229a32,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "540b7307-8fea-4b50-bab8-626983135a96", "address": "fa:16:3e:8d:a9:ab", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.135", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap540b7307-8f", "ovs_interfaceid": "540b7307-8fea-4b50-bab8-626983135a96", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Converting VIF {"id": "540b7307-8fea-4b50-bab8-626983135a96", "address": "fa:16:3e:8d:a9:ab", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.135", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap540b7307-8f", "ovs_interfaceid": "540b7307-8fea-4b50-bab8-626983135a96", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:8d:a9:ab,bridge_name='br-int',has_traffic_filtering=True,id=540b7307-8fea-4b50-bab8-626983135a96,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap540b7307-8f') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lazy-loading 'pci_devices' on Instance uuid b23d4bcd-989a-45e1-89a3-6e3d30229a32 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:34:12 user nova-compute[72081]:   <uuid>b23d4bcd-989a-45e1-89a3-6e3d30229a32</uuid>
Apr 12 03:34:12 user nova-compute[72081]:   <name>instance-00000003</name>
Apr 12 03:34:12 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:34:12 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:34:12 user nova-compute[72081]:   <metadata>
Apr 12 03:34:12 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:34:12 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:34:12 user nova-compute[72081]:       <nova:name>tempest-AttachVolumeShelveTestJSON-server-1925048093</nova:name>
Apr 12 03:34:12 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:34:10</nova:creationTime>
Apr 12 03:34:12 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:34:12 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:34:12 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:34:12 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:34:12 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:34:12 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:34:12 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:34:12 user nova-compute[72081]:       <nova:owner>
Apr 12 03:34:12 user nova-compute[72081]:         <nova:user uuid="de1fe5db01624224ab7b615c908d990b">tempest-AttachVolumeShelveTestJSON-2139975165-project-member</nova:user>
Apr 12 03:34:12 user nova-compute[72081]:         <nova:project uuid="738ebb6b798c4974aa46f9bbe3828279">tempest-AttachVolumeShelveTestJSON-2139975165</nova:project>
Apr 12 03:34:12 user nova-compute[72081]:       </nova:owner>
Apr 12 03:34:12 user nova-compute[72081]:       <nova:root type="image" uuid="1feda9ce-2630-410a-acd7-d4d319edb2af"/>
Apr 12 03:34:12 user nova-compute[72081]:       <nova:ports>
Apr 12 03:34:12 user nova-compute[72081]:         <nova:port uuid="540b7307-8fea-4b50-bab8-626983135a96">
Apr 12 03:34:12 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Apr 12 03:34:12 user nova-compute[72081]:         </nova:port>
Apr 12 03:34:12 user nova-compute[72081]:       </nova:ports>
Apr 12 03:34:12 user nova-compute[72081]:     </nova:instance>
Apr 12 03:34:12 user nova-compute[72081]:   </metadata>
Apr 12 03:34:12 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:34:12 user nova-compute[72081]:     <system>
Apr 12 03:34:12 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:34:12 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:34:12 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:34:12 user nova-compute[72081]:       <entry name="serial">b23d4bcd-989a-45e1-89a3-6e3d30229a32</entry>
Apr 12 03:34:12 user nova-compute[72081]:       <entry name="uuid">b23d4bcd-989a-45e1-89a3-6e3d30229a32</entry>
Apr 12 03:34:12 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:34:12 user nova-compute[72081]:     </system>
Apr 12 03:34:12 user nova-compute[72081]:   </sysinfo>
Apr 12 03:34:12 user nova-compute[72081]:   <os>
Apr 12 03:34:12 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:34:12 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:34:12 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:34:12 user nova-compute[72081]:   </os>
Apr 12 03:34:12 user nova-compute[72081]:   <features>
Apr 12 03:34:12 user nova-compute[72081]:     <acpi/>
Apr 12 03:34:12 user nova-compute[72081]:     <apic/>
Apr 12 03:34:12 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:34:12 user nova-compute[72081]:   </features>
Apr 12 03:34:12 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:34:12 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:34:12 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:34:12 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:34:12 user nova-compute[72081]:   </clock>
Apr 12 03:34:12 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:34:12 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:34:12 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:34:12 user nova-compute[72081]:   </cpu>
Apr 12 03:34:12 user nova-compute[72081]:   <devices>
Apr 12 03:34:12 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:34:12 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:34:12 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk"/>
Apr 12 03:34:12 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:34:12 user nova-compute[72081]:     </disk>
Apr 12 03:34:12 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:34:12 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:34:12 user nova-compute[72081]:       <source dev="/dev/sdd"/>
Apr 12 03:34:12 user nova-compute[72081]:       <target dev="vdc" bus="virtio"/>
Apr 12 03:34:12 user nova-compute[72081]:       <serial>e8b7fdb9-05e3-4870-b435-4c063881640f</serial>
Apr 12 03:34:12 user nova-compute[72081]:     </disk>
Apr 12 03:34:12 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:34:12 user nova-compute[72081]:       <mac address="fa:16:3e:8d:a9:ab"/>
Apr 12 03:34:12 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:34:12 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:34:12 user nova-compute[72081]:       <target dev="tap540b7307-8f"/>
Apr 12 03:34:12 user nova-compute[72081]:     </interface>
Apr 12 03:34:12 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:34:12 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/console.log" append="off"/>
Apr 12 03:34:12 user nova-compute[72081]:     </serial>
Apr 12 03:34:12 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:34:12 user nova-compute[72081]:     <video>
Apr 12 03:34:12 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:34:12 user nova-compute[72081]:     </video>
Apr 12 03:34:12 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:34:12 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:34:12 user nova-compute[72081]:     </rng>
Apr 12 03:34:12 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:34:12 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:34:12 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:34:12 user nova-compute[72081]:     </memballoon>
Apr 12 03:34:12 user nova-compute[72081]:   </devices>
Apr 12 03:34:12 user nova-compute[72081]: </domain>
Apr 12 03:34:12 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-12T03:30:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-1925048093',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-1925048093',id=3,image_ref='1feda9ce-2630-410a-acd7-d4d319edb2af',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name='tempest-keypair-892402082',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:30:30Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='738ebb6b798c4974aa46f9bbe3828279',ramdisk_id='',reservation_id='r-ybseqfps',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-2139975165',owner_user_name='tempest-AttachVolumeShelveTestJSON-2139975165-project-member',shelved_at='2023-04-12T03:32:09.253612',shelved_host='user',shelved_image_id='1feda9ce-2630-410a-acd7-d4d319edb2af'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:33:57Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='de1fe5db01624224ab7b615c908d990b',uuid=b23d4bcd-989a-45e1-89a3-6e3d30229a32,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "540b7307-8fea-4b50-bab8-626983135a96", "address": "fa:16:3e:8d:a9:ab", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.135", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap540b7307-8f", "ovs_interfaceid": "540b7307-8fea-4b50-bab8-626983135a96", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Converting VIF {"id": "540b7307-8fea-4b50-bab8-626983135a96", "address": "fa:16:3e:8d:a9:ab", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.135", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap540b7307-8f", "ovs_interfaceid": "540b7307-8fea-4b50-bab8-626983135a96", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:8d:a9:ab,bridge_name='br-int',has_traffic_filtering=True,id=540b7307-8fea-4b50-bab8-626983135a96,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap540b7307-8f') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG os_vif [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:8d:a9:ab,bridge_name='br-int',has_traffic_filtering=True,id=540b7307-8fea-4b50-bab8-626983135a96,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap540b7307-8f') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap540b7307-8f, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap540b7307-8f, col_values=(('external_ids', {'iface-id': '540b7307-8fea-4b50-bab8-626983135a96', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:8d:a9:ab', 'vm-uuid': 'b23d4bcd-989a-45e1-89a3-6e3d30229a32'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:12 user nova-compute[72081]: INFO os_vif [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:8d:a9:ab,bridge_name='br-int',has_traffic_filtering=True,id=540b7307-8fea-4b50-bab8-626983135a96,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap540b7307-8f')
Apr 12 03:34:12 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Updating instance_info_cache with network_info: [{"id": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "address": "fa:16:3e:c0:f6:ee", "network": {"id": "b4464ad0-6022-43e7-bfe4-229e6a3a0600", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1704232678-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.188", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a027d7708e6747fe970c388d6ba66359", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcff588d4-f4", "ovs_interfaceid": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] No BDM found with device name vdc, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] No VIF found with MAC fa:16:3e:8d:a9:ab, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Releasing lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG nova.compute.manager [req-77b3cd56-21a9-4892-9529-43833d94fb58 req-1eb31a29-a312-4469-abc1-ff2854739957 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received event network-vif-unplugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-77b3cd56-21a9-4892-9529-43833d94fb58 req-1eb31a29-a312-4469-abc1-ff2854739957 service nova] Acquiring lock "72ba1218-065d-40c4-8503-7c4155e47503-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-77b3cd56-21a9-4892-9529-43833d94fb58 req-1eb31a29-a312-4469-abc1-ff2854739957 service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-77b3cd56-21a9-4892-9529-43833d94fb58 req-1eb31a29-a312-4469-abc1-ff2854739957 service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG nova.compute.manager [req-77b3cd56-21a9-4892-9529-43833d94fb58 req-1eb31a29-a312-4469-abc1-ff2854739957 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] No waiting events found dispatching network-vif-unplugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:34:12 user nova-compute[72081]: WARNING nova.compute.manager [req-77b3cd56-21a9-4892-9529-43833d94fb58 req-1eb31a29-a312-4469-abc1-ff2854739957 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received unexpected event network-vif-unplugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 for instance with vm_state active and task_state rescuing.
Apr 12 03:34:12 user nova-compute[72081]: DEBUG nova.compute.manager [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:12 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:34:12 user nova-compute[72081]: INFO nova.compute.claims [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Claim successful on node user
Apr 12 03:34:13 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:34:13 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:34:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.475s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:13 user nova-compute[72081]: DEBUG nova.compute.manager [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:34:13 user nova-compute[72081]: DEBUG nova.compute.manager [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:34:13 user nova-compute[72081]: DEBUG nova.network.neutron [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:34:13 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:34:13 user nova-compute[72081]: DEBUG nova.compute.manager [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:34:13 user nova-compute[72081]: DEBUG nova.policy [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '84ea3a8776a944178ba5a04bce4d9b05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86f7b42b161f41efb234f0ce70210a08', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:34:13 user nova-compute[72081]: DEBUG nova.compute.manager [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:34:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:34:13 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Creating image(s)
Apr 12 03:34:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "/opt/stack/data/nova/instances/3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "/opt/stack/data/nova/instances/3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "/opt/stack/data/nova/instances/3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.009s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.148s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:13 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Instance destroyed successfully.
Apr 12 03:34:13 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Attempting a stable device rescue
Apr 12 03:34:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdc', 'type': 'disk'}}} {{(pid=72081) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4289}}
Apr 12 03:34:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Instance directory exists: not creating {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4694}}
Apr 12 03:34:13 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Creating image(s)
Apr 12 03:34:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Acquiring lock "/opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "/opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "/opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.003s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:13 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lazy-loading 'trusted_certs' on Instance uuid 72ba1218-065d-40c4-8503-7c4155e47503 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:34:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.157s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Acquiring lock "f8989bf22d0757204243efbf1bf4173532921efe" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "f8989bf22d0757204243efbf1bf4173532921efe" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.003s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f/disk 1073741824" returned: 0 in 0.053s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.214s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f8989bf22d0757204243efbf1bf4173532921efe.part --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG nova.network.neutron [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Successfully created port: e3161de1-7643-4b1a-8a0c-27791fed8dbd {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG nova.compute.manager [req-f3da2e56-79f7-4563-b244-8fea1d8b5a3d req-b52b4be4-658b-4729-943a-ff1b237f656c service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received event network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f3da2e56-79f7-4563-b244-8fea1d8b5a3d req-b52b4be4-658b-4729-943a-ff1b237f656c service nova] Acquiring lock "72ba1218-065d-40c4-8503-7c4155e47503-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f3da2e56-79f7-4563-b244-8fea1d8b5a3d req-b52b4be4-658b-4729-943a-ff1b237f656c service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f3da2e56-79f7-4563-b244-8fea1d8b5a3d req-b52b4be4-658b-4729-943a-ff1b237f656c service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG nova.compute.manager [req-f3da2e56-79f7-4563-b244-8fea1d8b5a3d req-b52b4be4-658b-4729-943a-ff1b237f656c service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] No waiting events found dispatching network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:34:15 user nova-compute[72081]: WARNING nova.compute.manager [req-f3da2e56-79f7-4563-b244-8fea1d8b5a3d req-b52b4be4-658b-4729-943a-ff1b237f656c service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received unexpected event network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 for instance with vm_state active and task_state rescuing.
Apr 12 03:34:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 1.062s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Checking if we can resize image /opt/stack/data/nova/instances/3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f8989bf22d0757204243efbf1bf4173532921efe.part --force-share --output=json" returned: 0 in 0.227s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG nova.virt.images [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] 3edad615-4b98-4e15-bf3e-72fa0b622eca was qcow2, converting to raw {{(pid=72081) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG nova.privsep.utils [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72081) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/f8989bf22d0757204243efbf1bf4173532921efe.part /opt/stack/data/nova/instances/_base/f8989bf22d0757204243efbf1bf4173532921efe.converted {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f/disk --force-share --output=json" returned: 0 in 0.180s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Cannot resize image /opt/stack/data/nova/instances/3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG nova.objects.instance [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lazy-loading 'migration_context' on Instance uuid 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Ensure instance console log exists: /opt/stack/data/nova/instances/3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG nova.compute.manager [req-56d6f026-61c9-44f2-8428-88d32ab40706 req-b707047a-efe1-4da4-9e75-68c3ee9f95d6 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Received event network-vif-plugged-540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-56d6f026-61c9-44f2-8428-88d32ab40706 req-b707047a-efe1-4da4-9e75-68c3ee9f95d6 service nova] Acquiring lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-56d6f026-61c9-44f2-8428-88d32ab40706 req-b707047a-efe1-4da4-9e75-68c3ee9f95d6 service nova] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-56d6f026-61c9-44f2-8428-88d32ab40706 req-b707047a-efe1-4da4-9e75-68c3ee9f95d6 service nova] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG nova.compute.manager [req-56d6f026-61c9-44f2-8428-88d32ab40706 req-b707047a-efe1-4da4-9e75-68c3ee9f95d6 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] No waiting events found dispatching network-vif-plugged-540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:34:15 user nova-compute[72081]: WARNING nova.compute.manager [req-56d6f026-61c9-44f2-8428-88d32ab40706 req-b707047a-efe1-4da4-9e75-68c3ee9f95d6 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Received unexpected event network-vif-plugged-540b7307-8fea-4b50-bab8-626983135a96 for instance with vm_state shelved_offloaded and task_state spawning.
Apr 12 03:34:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/f8989bf22d0757204243efbf1bf4173532921efe.part /opt/stack/data/nova/instances/_base/f8989bf22d0757204243efbf1bf4173532921efe.converted" returned: 0 in 0.421s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f8989bf22d0757204243efbf1bf4173532921efe.converted --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f8989bf22d0757204243efbf1bf4173532921efe.converted --force-share --output=json" returned: 0 in 0.154s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "f8989bf22d0757204243efbf1bf4173532921efe" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.828s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Acquiring lock "f8989bf22d0757204243efbf1bf4173532921efe" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "f8989bf22d0757204243efbf1bf4173532921efe" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f8989bf22d0757204243efbf1bf4173532921efe --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f8989bf22d0757204243efbf1bf4173532921efe --force-share --output=json" returned: 0 in 0.131s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f8989bf22d0757204243efbf1bf4173532921efe,backing_fmt=raw /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk.rescue {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f8989bf22d0757204243efbf1bf4173532921efe,backing_fmt=raw /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk.rescue" returned: 0 in 0.068s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "f8989bf22d0757204243efbf1bf4173532921efe" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.213s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lazy-loading 'migration_context' on Instance uuid 72ba1218-065d-40c4-8503-7c4155e47503 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Start _get_guest_xml network_info=[{"id": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "address": "fa:16:3e:c0:f6:ee", "network": {"id": "b4464ad0-6022-43e7-bfe4-229e6a3a0600", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1704232678-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.188", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerStableDeviceRescueTest-1704232678-network", "vif_mac": "fa:16:3e:c0:f6:ee"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a027d7708e6747fe970c388d6ba66359", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcff588d4-f4", "ovs_interfaceid": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdc', 'type': 'disk'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': '3edad615-4b98-4e15-bf3e-72fa0b622eca', 'kernel_id': '', 'ramdisk_id': ''} block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vdb', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'VLuEwIle', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, 'status': 'reserved', 'instance': '72ba1218-065d-40c4-8503-7c4155e47503', 'attached_at': '', 'detached_at': '', 'volume_id': 'd2e08409-6c4e-44ef-842b-29460b0cb939', 'serial': 'd2e08409-6c4e-44ef-842b-29460b0cb939'}, 'disk_bus': 'virtio', 'delete_on_termination': False, 'boot_index': None, 'guest_format': None, 'device_type': 'disk', 'attachment_id': '73131876-1db8-4d30-9361-a770cf229ba1', 'volume_type': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lazy-loading 'resources' on Instance uuid 72ba1218-065d-40c4-8503-7c4155e47503 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lazy-loading 'numa_topology' on Instance uuid 72ba1218-065d-40c4-8503-7c4155e47503 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:34:16 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:34:16 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lazy-loading 'vcpu_model' on Instance uuid 72ba1218-065d-40c4-8503-7c4155e47503 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'VLuEwIle', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:34:16 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:34:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:34:16 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:34:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:34:16 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:34:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4de69f66-753a-4682-b1b1-31f81bca7491]: (4, ('# BEGIN RECORD 2.1.5\nnode.name = iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = <empty>\niface.ipaddress = <empty>\niface.prefix_len = 0\niface.hwaddress = <empty>\niface.transport_name = tcp\niface.initiatorname = <empty>\niface.state = <empty>\niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = <empty>\niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = <empty>\niface.subnet_mask = <empty>\niface.gateway = <empty>\niface.dhcp_alt_client_id_state = <empty>\niface.dhcp_alt_client_id = <empty>\niface.dhcp_dns = <empty>\niface.dhcp_learn_iqn = <empty>\niface.dhcp_req_vendor_id_state = <empty>\niface.dhcp_vendor_id_state = <empty>\niface.dhcp_vendor_id = <empty>\niface.dhcp_slp_da = <empty>\niface.fragmentation = <empty>\niface.gratuitous_arp = <empty>\niface.incoming_forwarding = <empty>\niface.tos_state = <empty>\niface.tos = 0\niface.ttl = 0\niface.delayed_ack = <empty>\niface.tcp_nagle = <empty>\niface.tcp_wsf_state = <empty>\niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = <empty>\niface.redirect = <empty>\niface.def_task_mgmt_timeout = 0\niface.header_digest = <empty>\niface.data_digest = <empty>\niface.immediate_data = <empty>\niface.initial_r2t = <empty>\niface.data_seq_inorder = <empty>\niface.data_pdu_inorder = <empty>\niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = <empty>\niface.bidi_chap = <empty>\niface.strict_login_compliance = <empty>\niface.discovery_auth = <empty>\niface.discovery_logout = <empty>\nnode.discovery_address = <empty>\nnode.discovery_port = 0\nnode.discovery_type = static\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = -20\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = CHAP\nnode.session.auth.username = VLuEwIle\nnode.session.auth.password = ********\nnode.session.auth.username_in = <empty>\nnode.session.auth.password_in = <empty>\nnode.session.auth.chap_algs = MD5\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = manual\nnode.session.reopen_max = 0\nnode.conn[0].address = 172.16.0.220\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] iscsiadm (): stdout=# BEGIN RECORD 2.1.5
Apr 12 03:34:16 user nova-compute[72081]: node.name = iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939
Apr 12 03:34:16 user nova-compute[72081]: node.tpgt = -1
Apr 12 03:34:16 user nova-compute[72081]: node.startup = automatic
Apr 12 03:34:16 user nova-compute[72081]: node.leading_login = No
Apr 12 03:34:16 user nova-compute[72081]: iface.iscsi_ifacename = default
Apr 12 03:34:16 user nova-compute[72081]: iface.net_ifacename = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.ipaddress = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.prefix_len = 0
Apr 12 03:34:16 user nova-compute[72081]: iface.hwaddress = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.transport_name = tcp
Apr 12 03:34:16 user nova-compute[72081]: iface.initiatorname = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.state = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.vlan_id = 0
Apr 12 03:34:16 user nova-compute[72081]: iface.vlan_priority = 0
Apr 12 03:34:16 user nova-compute[72081]: iface.vlan_state = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.iface_num = 0
Apr 12 03:34:16 user nova-compute[72081]: iface.mtu = 0
Apr 12 03:34:16 user nova-compute[72081]: iface.port = 0
Apr 12 03:34:16 user nova-compute[72081]: iface.bootproto = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.subnet_mask = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.gateway = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.dhcp_alt_client_id_state = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.dhcp_alt_client_id = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.dhcp_dns = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.dhcp_learn_iqn = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.dhcp_req_vendor_id_state = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.dhcp_vendor_id_state = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.dhcp_vendor_id = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.dhcp_slp_da = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.fragmentation = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.gratuitous_arp = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.incoming_forwarding = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.tos_state = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.tos = 0
Apr 12 03:34:16 user nova-compute[72081]: iface.ttl = 0
Apr 12 03:34:16 user nova-compute[72081]: iface.delayed_ack = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.tcp_nagle = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.tcp_wsf_state = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.tcp_wsf = 0
Apr 12 03:34:16 user nova-compute[72081]: iface.tcp_timer_scale = 0
Apr 12 03:34:16 user nova-compute[72081]: iface.tcp_timestamp = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.redirect = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.def_task_mgmt_timeout = 0
Apr 12 03:34:16 user nova-compute[72081]: iface.header_digest = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.data_digest = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.immediate_data = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.initial_r2t = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.data_seq_inorder = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.data_pdu_inorder = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.erl = 0
Apr 12 03:34:16 user nova-compute[72081]: iface.max_receive_data_len = 0
Apr 12 03:34:16 user nova-compute[72081]: iface.first_burst_len = 0
Apr 12 03:34:16 user nova-compute[72081]: iface.max_outstanding_r2t = 0
Apr 12 03:34:16 user nova-compute[72081]: iface.max_burst_len = 0
Apr 12 03:34:16 user nova-compute[72081]: iface.chap_auth = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.bidi_chap = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.strict_login_compliance = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.discovery_auth = <empty>
Apr 12 03:34:16 user nova-compute[72081]: iface.discovery_logout = <empty>
Apr 12 03:34:16 user nova-compute[72081]: node.discovery_address = <empty>
Apr 12 03:34:16 user nova-compute[72081]: node.discovery_port = 0
Apr 12 03:34:16 user nova-compute[72081]: node.discovery_type = static
Apr 12 03:34:16 user nova-compute[72081]: node.session.initial_cmdsn = 0
Apr 12 03:34:16 user nova-compute[72081]: node.session.initial_login_retry_max = 8
Apr 12 03:34:16 user nova-compute[72081]: node.session.xmit_thread_priority = -20
Apr 12 03:34:16 user nova-compute[72081]: node.session.cmds_max = 128
Apr 12 03:34:16 user nova-compute[72081]: node.session.queue_depth = 32
Apr 12 03:34:16 user nova-compute[72081]: node.session.nr_sessions = 1
Apr 12 03:34:16 user nova-compute[72081]: node.session.auth.authmethod = CHAP
Apr 12 03:34:16 user nova-compute[72081]: node.session.auth.username = VLuEwIle
Apr 12 03:34:16 user nova-compute[72081]: node.session.auth.password = ***
Apr 12 03:34:16 user nova-compute[72081]: node.session.auth.username_in = <empty>
Apr 12 03:34:16 user nova-compute[72081]: node.session.auth.password_in = <empty>
Apr 12 03:34:16 user nova-compute[72081]: node.session.auth.chap_algs = MD5
Apr 12 03:34:16 user nova-compute[72081]: node.session.timeo.replacement_timeout = 120
Apr 12 03:34:16 user nova-compute[72081]: node.session.err_timeo.abort_timeout = 15
Apr 12 03:34:16 user nova-compute[72081]: node.session.err_timeo.lu_reset_timeout = 30
Apr 12 03:34:16 user nova-compute[72081]: node.session.err_timeo.tgt_reset_timeout = 30
Apr 12 03:34:16 user nova-compute[72081]: node.session.err_timeo.host_reset_timeout = 60
Apr 12 03:34:16 user nova-compute[72081]: node.session.iscsi.FastAbort = Yes
Apr 12 03:34:16 user nova-compute[72081]: node.session.iscsi.InitialR2T = No
Apr 12 03:34:16 user nova-compute[72081]: node.session.iscsi.ImmediateData = Yes
Apr 12 03:34:16 user nova-compute[72081]: node.session.iscsi.FirstBurstLength = 262144
Apr 12 03:34:16 user nova-compute[72081]: node.session.iscsi.MaxBurstLength = 16776192
Apr 12 03:34:16 user nova-compute[72081]: node.session.iscsi.DefaultTime2Retain = 0
Apr 12 03:34:16 user nova-compute[72081]: node.session.iscsi.DefaultTime2Wait = 2
Apr 12 03:34:16 user nova-compute[72081]: node.session.iscsi.MaxConnections = 1
Apr 12 03:34:16 user nova-compute[72081]: node.session.iscsi.MaxOutstandingR2T = 1
Apr 12 03:34:16 user nova-compute[72081]: node.session.iscsi.ERL = 0
Apr 12 03:34:16 user nova-compute[72081]: node.session.scan = manual
Apr 12 03:34:16 user nova-compute[72081]: node.session.reopen_max = 0
Apr 12 03:34:16 user nova-compute[72081]: node.conn[0].address = 172.16.0.220
Apr 12 03:34:16 user nova-compute[72081]: node.conn[0].port = 3260
Apr 12 03:34:16 user nova-compute[72081]: node.conn[0].startup = manual
Apr 12 03:34:16 user nova-compute[72081]: node.conn[0].tcp.window_size = 524288
Apr 12 03:34:16 user nova-compute[72081]: node.conn[0].tcp.type_of_service = 0
Apr 12 03:34:16 user nova-compute[72081]: node.conn[0].timeo.logout_timeout = 15
Apr 12 03:34:16 user nova-compute[72081]: node.conn[0].timeo.login_timeout = 15
Apr 12 03:34:16 user nova-compute[72081]: node.conn[0].timeo.auth_timeout = 45
Apr 12 03:34:16 user nova-compute[72081]: node.conn[0].timeo.noop_out_interval = 5
Apr 12 03:34:16 user nova-compute[72081]: node.conn[0].timeo.noop_out_timeout = 5
Apr 12 03:34:16 user nova-compute[72081]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0
Apr 12 03:34:16 user nova-compute[72081]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144
Apr 12 03:34:16 user nova-compute[72081]: node.conn[0].iscsi.HeaderDigest = None
Apr 12 03:34:16 user nova-compute[72081]: node.conn[0].iscsi.DataDigest = None
Apr 12 03:34:16 user nova-compute[72081]: node.conn[0].iscsi.IFMarker = No
Apr 12 03:34:16 user nova-compute[72081]: node.conn[0].iscsi.OFMarker = No
Apr 12 03:34:16 user nova-compute[72081]: # END RECORD
Apr 12 03:34:16 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4cdf3f66-0a45-46ff-91b8-34549d73234f]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.034s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[64931f24-3014-4437-a6aa-e572f05c59e4]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v VLuEwIle {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v VLuEwIle" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[db300f38-040f-4e4a-9c73-60ee1a7e4efd]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'VLuEwIle'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c47f8ff8-4181-41d8-816f-6c544f05b777]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.035s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[caadccd4-4bee-4e7d-8af1-dd8ac3a3ac64]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:34:16 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:34:16 user nova-compute[72081]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f (non-flash)
Apr 12 03:34:16 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:34:16 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:34:16 user nova-compute[72081]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f (non-flash)
Apr 12 03:34:16 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.209s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] HCTL ('34', '0', '0', 0) found on session 15 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Scanning host 34 c: 0, t: 0, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2b9dfde5-b35d-4d5a-96d6-0b67139858c2]: (4, ('0 0 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Searching for a device in session 15 and hctl ('34', '0', '0', 0) yield: sdc {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'VLuEwIle', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 0.242s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] <== connect_volume: return (244ms) {'type': 'block', 'scsi_wwn': '23735336265623431', 'path': '/dev/sdc'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23735336265623431', 'path': '/dev/sdc'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:30:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerStableDeviceRescueTest-server-252473144',display_name='tempest-ServerStableDeviceRescueTest-server-252473144',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverstabledevicerescuetest-server-252473144',id=5,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBInWJADLpni9D9TfgnC1+/LMPrrQuD/lxuqixLR+mL8FYE3064UBREFALqAw20h2nHoOSsmv2cMRVuFj3Oi+UewdLxZXOi4pdekw/Cvt3tr0uQWDcvZgNGpzrP2V/qty5w==',key_name='tempest-keypair-31851495',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:30:33Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='a027d7708e6747fe970c388d6ba66359',ramdisk_id='',reservation_id='r-30kyz70u',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerStableDeviceRescueTest-744520899',owner_user_name='tempest-ServerStableDeviceRescueTest-744520899-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:32:13Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='9405e57e55e84976b0caeba84e3aef62',uuid=72ba1218-065d-40c4-8503-7c4155e47503,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "address": "fa:16:3e:c0:f6:ee", "network": {"id": "b4464ad0-6022-43e7-bfe4-229e6a3a0600", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1704232678-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.188", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerStableDeviceRescueTest-1704232678-network", "vif_mac": "fa:16:3e:c0:f6:ee"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a027d7708e6747fe970c388d6ba66359", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcff588d4-f4", "ovs_interfaceid": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Converting VIF {"id": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "address": "fa:16:3e:c0:f6:ee", "network": {"id": "b4464ad0-6022-43e7-bfe4-229e6a3a0600", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1704232678-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.188", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerStableDeviceRescueTest-1704232678-network", "vif_mac": "fa:16:3e:c0:f6:ee"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a027d7708e6747fe970c388d6ba66359", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcff588d4-f4", "ovs_interfaceid": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:c0:f6:ee,bridge_name='br-int',has_traffic_filtering=True,id=cff588d4-f46a-4d44-a3e6-1c04f6abd380,network=Network(b4464ad0-6022-43e7-bfe4-229e6a3a0600),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapcff588d4-f4') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lazy-loading 'pci_devices' on Instance uuid 72ba1218-065d-40c4-8503-7c4155e47503 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:34:16 user nova-compute[72081]:   <uuid>72ba1218-065d-40c4-8503-7c4155e47503</uuid>
Apr 12 03:34:16 user nova-compute[72081]:   <name>instance-00000005</name>
Apr 12 03:34:16 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:34:16 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:34:16 user nova-compute[72081]:   <metadata>
Apr 12 03:34:16 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:34:16 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:34:16 user nova-compute[72081]:       <nova:name>tempest-ServerStableDeviceRescueTest-server-252473144</nova:name>
Apr 12 03:34:16 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:34:16</nova:creationTime>
Apr 12 03:34:16 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:34:16 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:34:16 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:34:16 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:34:16 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:34:16 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:34:16 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:34:16 user nova-compute[72081]:       <nova:owner>
Apr 12 03:34:16 user nova-compute[72081]:         <nova:user uuid="9405e57e55e84976b0caeba84e3aef62">tempest-ServerStableDeviceRescueTest-744520899-project-member</nova:user>
Apr 12 03:34:16 user nova-compute[72081]:         <nova:project uuid="a027d7708e6747fe970c388d6ba66359">tempest-ServerStableDeviceRescueTest-744520899</nova:project>
Apr 12 03:34:16 user nova-compute[72081]:       </nova:owner>
Apr 12 03:34:16 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:34:16 user nova-compute[72081]:       <nova:ports>
Apr 12 03:34:16 user nova-compute[72081]:         <nova:port uuid="cff588d4-f46a-4d44-a3e6-1c04f6abd380">
Apr 12 03:34:16 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Apr 12 03:34:16 user nova-compute[72081]:         </nova:port>
Apr 12 03:34:16 user nova-compute[72081]:       </nova:ports>
Apr 12 03:34:16 user nova-compute[72081]:     </nova:instance>
Apr 12 03:34:16 user nova-compute[72081]:   </metadata>
Apr 12 03:34:16 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:34:16 user nova-compute[72081]:     <system>
Apr 12 03:34:16 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:34:16 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:34:16 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:34:16 user nova-compute[72081]:       <entry name="serial">72ba1218-065d-40c4-8503-7c4155e47503</entry>
Apr 12 03:34:16 user nova-compute[72081]:       <entry name="uuid">72ba1218-065d-40c4-8503-7c4155e47503</entry>
Apr 12 03:34:16 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:34:16 user nova-compute[72081]:     </system>
Apr 12 03:34:16 user nova-compute[72081]:   </sysinfo>
Apr 12 03:34:16 user nova-compute[72081]:   <os>
Apr 12 03:34:16 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:34:16 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:34:16 user nova-compute[72081]:   </os>
Apr 12 03:34:16 user nova-compute[72081]:   <features>
Apr 12 03:34:16 user nova-compute[72081]:     <acpi/>
Apr 12 03:34:16 user nova-compute[72081]:     <apic/>
Apr 12 03:34:16 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:34:16 user nova-compute[72081]:   </features>
Apr 12 03:34:16 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:34:16 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:34:16 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:34:16 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:34:16 user nova-compute[72081]:   </clock>
Apr 12 03:34:16 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:34:16 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:34:16 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:34:16 user nova-compute[72081]:   </cpu>
Apr 12 03:34:16 user nova-compute[72081]:   <devices>
Apr 12 03:34:16 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:34:16 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:34:16 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk"/>
Apr 12 03:34:16 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:34:16 user nova-compute[72081]:     </disk>
Apr 12 03:34:16 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:34:16 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:34:16 user nova-compute[72081]:       <source dev="/dev/sdc"/>
Apr 12 03:34:16 user nova-compute[72081]:       <target dev="vdb" bus="virtio"/>
Apr 12 03:34:16 user nova-compute[72081]:       <serial>d2e08409-6c4e-44ef-842b-29460b0cb939</serial>
Apr 12 03:34:16 user nova-compute[72081]:     </disk>
Apr 12 03:34:16 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:34:16 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:34:16 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk.rescue"/>
Apr 12 03:34:16 user nova-compute[72081]:       <target dev="vdc" bus="virtio"/>
Apr 12 03:34:16 user nova-compute[72081]:       <boot order="1"/>
Apr 12 03:34:16 user nova-compute[72081]:     </disk>
Apr 12 03:34:16 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:34:16 user nova-compute[72081]:       <mac address="fa:16:3e:c0:f6:ee"/>
Apr 12 03:34:16 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:34:16 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:34:16 user nova-compute[72081]:       <target dev="tapcff588d4-f4"/>
Apr 12 03:34:16 user nova-compute[72081]:     </interface>
Apr 12 03:34:16 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:34:16 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/console.log" append="off"/>
Apr 12 03:34:16 user nova-compute[72081]:     </serial>
Apr 12 03:34:16 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:34:16 user nova-compute[72081]:     <video>
Apr 12 03:34:16 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:34:16 user nova-compute[72081]:     </video>
Apr 12 03:34:16 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:34:16 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:34:16 user nova-compute[72081]:     </rng>
Apr 12 03:34:16 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:34:16 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:34:16 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:34:16 user nova-compute[72081]:     </memballoon>
Apr 12 03:34:16 user nova-compute[72081]:   </devices>
Apr 12 03:34:16 user nova-compute[72081]: </domain>
Apr 12 03:34:16 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:34:16 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Instance destroyed successfully.
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] No BDM found with device name vdc, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:34:16 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] No VIF found with MAC fa:16:3e:c0:f6:ee, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.compute.manager [req-12b13804-d5c6-4534-b609-62afae036a34 req-c1184c9e-576d-4529-93a6-8df4b2d9a08a service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Received event network-vif-plugged-b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-12b13804-d5c6-4534-b609-62afae036a34 req-c1184c9e-576d-4529-93a6-8df4b2d9a08a service nova] Acquiring lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-12b13804-d5c6-4534-b609-62afae036a34 req-c1184c9e-576d-4529-93a6-8df4b2d9a08a service nova] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-12b13804-d5c6-4534-b609-62afae036a34 req-c1184c9e-576d-4529-93a6-8df4b2d9a08a service nova] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.compute.manager [req-12b13804-d5c6-4534-b609-62afae036a34 req-c1184c9e-576d-4529-93a6-8df4b2d9a08a service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] No waiting events found dispatching network-vif-plugged-b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:34:17 user nova-compute[72081]: WARNING nova.compute.manager [req-12b13804-d5c6-4534-b609-62afae036a34 req-c1184c9e-576d-4529-93a6-8df4b2d9a08a service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Received unexpected event network-vif-plugged-b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958 for instance with vm_state building and task_state spawning.
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.compute.manager [req-12b13804-d5c6-4534-b609-62afae036a34 req-c1184c9e-576d-4529-93a6-8df4b2d9a08a service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Received event network-vif-plugged-b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-12b13804-d5c6-4534-b609-62afae036a34 req-c1184c9e-576d-4529-93a6-8df4b2d9a08a service nova] Acquiring lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-12b13804-d5c6-4534-b609-62afae036a34 req-c1184c9e-576d-4529-93a6-8df4b2d9a08a service nova] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-12b13804-d5c6-4534-b609-62afae036a34 req-c1184c9e-576d-4529-93a6-8df4b2d9a08a service nova] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.compute.manager [req-12b13804-d5c6-4534-b609-62afae036a34 req-c1184c9e-576d-4529-93a6-8df4b2d9a08a service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] No waiting events found dispatching network-vif-plugged-b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:34:17 user nova-compute[72081]: WARNING nova.compute.manager [req-12b13804-d5c6-4534-b609-62afae036a34 req-c1184c9e-576d-4529-93a6-8df4b2d9a08a service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Received unexpected event network-vif-plugged-b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958 for instance with vm_state building and task_state spawning.
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.network.neutron [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Successfully updated port: e3161de1-7643-4b1a-8a0c-27791fed8dbd {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "refresh_cache-3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquired lock "refresh_cache-3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.network.neutron [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.network.neutron [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.compute.manager [req-7dc7d582-a1e7-449d-ad15-8e6c58937051 req-526b627b-742d-4321-85f6-0ac95b877132 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Received event network-vif-plugged-540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7dc7d582-a1e7-449d-ad15-8e6c58937051 req-526b627b-742d-4321-85f6-0ac95b877132 service nova] Acquiring lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7dc7d582-a1e7-449d-ad15-8e6c58937051 req-526b627b-742d-4321-85f6-0ac95b877132 service nova] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7dc7d582-a1e7-449d-ad15-8e6c58937051 req-526b627b-742d-4321-85f6-0ac95b877132 service nova] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.compute.manager [req-7dc7d582-a1e7-449d-ad15-8e6c58937051 req-526b627b-742d-4321-85f6-0ac95b877132 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] No waiting events found dispatching network-vif-plugged-540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:34:17 user nova-compute[72081]: WARNING nova.compute.manager [req-7dc7d582-a1e7-449d-ad15-8e6c58937051 req-526b627b-742d-4321-85f6-0ac95b877132 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Received unexpected event network-vif-plugged-540b7307-8fea-4b50-bab8-626983135a96 for instance with vm_state shelved_offloaded and task_state spawning.
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.network.neutron [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Updating instance_info_cache with network_info: [{"id": "e3161de1-7643-4b1a-8a0c-27791fed8dbd", "address": "fa:16:3e:71:14:95", "network": {"id": "07d0f97b-06f9-40b3-8794-e53924379d25", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-2076424457-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "86f7b42b161f41efb234f0ce70210a08", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3161de1-76", "ovs_interfaceid": "e3161de1-7643-4b1a-8a0c-27791fed8dbd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Releasing lock "refresh_cache-3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.compute.manager [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Instance network_info: |[{"id": "e3161de1-7643-4b1a-8a0c-27791fed8dbd", "address": "fa:16:3e:71:14:95", "network": {"id": "07d0f97b-06f9-40b3-8794-e53924379d25", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-2076424457-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "86f7b42b161f41efb234f0ce70210a08", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3161de1-76", "ovs_interfaceid": "e3161de1-7643-4b1a-8a0c-27791fed8dbd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Start _get_guest_xml network_info=[{"id": "e3161de1-7643-4b1a-8a0c-27791fed8dbd", "address": "fa:16:3e:71:14:95", "network": {"id": "07d0f97b-06f9-40b3-8794-e53924379d25", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-2076424457-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "86f7b42b161f41efb234f0ce70210a08", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3161de1-76", "ovs_interfaceid": "e3161de1-7643-4b1a-8a0c-27791fed8dbd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:34:17 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:34:17 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:34:12Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-99867029',display_name='tempest-VolumesSnapshotTestJSON-instance-99867029',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-99867029',id=16,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='86f7b42b161f41efb234f0ce70210a08',ramdisk_id='',reservation_id='r-r7onnebb',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesSnapshotTestJSON-1513599190',owner_user_name='tempest-VolumesSnapshotTestJSON-1513599190-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:34:14Z,user_data=None,user_id='84ea3a8776a944178ba5a04bce4d9b05',uuid=3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e3161de1-7643-4b1a-8a0c-27791fed8dbd", "address": "fa:16:3e:71:14:95", "network": {"id": "07d0f97b-06f9-40b3-8794-e53924379d25", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-2076424457-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "86f7b42b161f41efb234f0ce70210a08", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3161de1-76", "ovs_interfaceid": "e3161de1-7643-4b1a-8a0c-27791fed8dbd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Converting VIF {"id": "e3161de1-7643-4b1a-8a0c-27791fed8dbd", "address": "fa:16:3e:71:14:95", "network": {"id": "07d0f97b-06f9-40b3-8794-e53924379d25", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-2076424457-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "86f7b42b161f41efb234f0ce70210a08", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3161de1-76", "ovs_interfaceid": "e3161de1-7643-4b1a-8a0c-27791fed8dbd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:71:14:95,bridge_name='br-int',has_traffic_filtering=True,id=e3161de1-7643-4b1a-8a0c-27791fed8dbd,network=Network(07d0f97b-06f9-40b3-8794-e53924379d25),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape3161de1-76') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.objects.instance [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lazy-loading 'pci_devices' on Instance uuid 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:34:17 user nova-compute[72081]:   <uuid>3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f</uuid>
Apr 12 03:34:17 user nova-compute[72081]:   <name>instance-00000010</name>
Apr 12 03:34:17 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:34:17 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:34:17 user nova-compute[72081]:   <metadata>
Apr 12 03:34:17 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:34:17 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:34:17 user nova-compute[72081]:       <nova:name>tempest-VolumesSnapshotTestJSON-instance-99867029</nova:name>
Apr 12 03:34:17 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:34:17</nova:creationTime>
Apr 12 03:34:17 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:34:17 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:34:17 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:34:17 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:34:17 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:34:17 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:34:17 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:34:17 user nova-compute[72081]:       <nova:owner>
Apr 12 03:34:17 user nova-compute[72081]:         <nova:user uuid="84ea3a8776a944178ba5a04bce4d9b05">tempest-VolumesSnapshotTestJSON-1513599190-project-member</nova:user>
Apr 12 03:34:17 user nova-compute[72081]:         <nova:project uuid="86f7b42b161f41efb234f0ce70210a08">tempest-VolumesSnapshotTestJSON-1513599190</nova:project>
Apr 12 03:34:17 user nova-compute[72081]:       </nova:owner>
Apr 12 03:34:17 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:34:17 user nova-compute[72081]:       <nova:ports>
Apr 12 03:34:17 user nova-compute[72081]:         <nova:port uuid="e3161de1-7643-4b1a-8a0c-27791fed8dbd">
Apr 12 03:34:17 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 12 03:34:17 user nova-compute[72081]:         </nova:port>
Apr 12 03:34:17 user nova-compute[72081]:       </nova:ports>
Apr 12 03:34:17 user nova-compute[72081]:     </nova:instance>
Apr 12 03:34:17 user nova-compute[72081]:   </metadata>
Apr 12 03:34:17 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:34:17 user nova-compute[72081]:     <system>
Apr 12 03:34:17 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:34:17 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:34:17 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:34:17 user nova-compute[72081]:       <entry name="serial">3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f</entry>
Apr 12 03:34:17 user nova-compute[72081]:       <entry name="uuid">3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f</entry>
Apr 12 03:34:17 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:34:17 user nova-compute[72081]:     </system>
Apr 12 03:34:17 user nova-compute[72081]:   </sysinfo>
Apr 12 03:34:17 user nova-compute[72081]:   <os>
Apr 12 03:34:17 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:34:17 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:34:17 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:34:17 user nova-compute[72081]:   </os>
Apr 12 03:34:17 user nova-compute[72081]:   <features>
Apr 12 03:34:17 user nova-compute[72081]:     <acpi/>
Apr 12 03:34:17 user nova-compute[72081]:     <apic/>
Apr 12 03:34:17 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:34:17 user nova-compute[72081]:   </features>
Apr 12 03:34:17 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:34:17 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:34:17 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:34:17 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:34:17 user nova-compute[72081]:   </clock>
Apr 12 03:34:17 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:34:17 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:34:17 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:34:17 user nova-compute[72081]:   </cpu>
Apr 12 03:34:17 user nova-compute[72081]:   <devices>
Apr 12 03:34:17 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:34:17 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:34:17 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f/disk"/>
Apr 12 03:34:17 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:34:17 user nova-compute[72081]:     </disk>
Apr 12 03:34:17 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:34:17 user nova-compute[72081]:       <mac address="fa:16:3e:71:14:95"/>
Apr 12 03:34:17 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:34:17 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:34:17 user nova-compute[72081]:       <target dev="tape3161de1-76"/>
Apr 12 03:34:17 user nova-compute[72081]:     </interface>
Apr 12 03:34:17 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:34:17 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f/console.log" append="off"/>
Apr 12 03:34:17 user nova-compute[72081]:     </serial>
Apr 12 03:34:17 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:34:17 user nova-compute[72081]:     <video>
Apr 12 03:34:17 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:34:17 user nova-compute[72081]:     </video>
Apr 12 03:34:17 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:34:17 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:34:17 user nova-compute[72081]:     </rng>
Apr 12 03:34:17 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:34:17 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:34:17 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:34:17 user nova-compute[72081]:     </memballoon>
Apr 12 03:34:17 user nova-compute[72081]:   </devices>
Apr 12 03:34:17 user nova-compute[72081]: </domain>
Apr 12 03:34:17 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:34:12Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-99867029',display_name='tempest-VolumesSnapshotTestJSON-instance-99867029',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-99867029',id=16,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='86f7b42b161f41efb234f0ce70210a08',ramdisk_id='',reservation_id='r-r7onnebb',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesSnapshotTestJSON-1513599190',owner_user_name='tempest-VolumesSnapshotTestJSON-1513599190-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:34:14Z,user_data=None,user_id='84ea3a8776a944178ba5a04bce4d9b05',uuid=3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e3161de1-7643-4b1a-8a0c-27791fed8dbd", "address": "fa:16:3e:71:14:95", "network": {"id": "07d0f97b-06f9-40b3-8794-e53924379d25", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-2076424457-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "86f7b42b161f41efb234f0ce70210a08", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3161de1-76", "ovs_interfaceid": "e3161de1-7643-4b1a-8a0c-27791fed8dbd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Converting VIF {"id": "e3161de1-7643-4b1a-8a0c-27791fed8dbd", "address": "fa:16:3e:71:14:95", "network": {"id": "07d0f97b-06f9-40b3-8794-e53924379d25", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-2076424457-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "86f7b42b161f41efb234f0ce70210a08", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3161de1-76", "ovs_interfaceid": "e3161de1-7643-4b1a-8a0c-27791fed8dbd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:71:14:95,bridge_name='br-int',has_traffic_filtering=True,id=e3161de1-7643-4b1a-8a0c-27791fed8dbd,network=Network(07d0f97b-06f9-40b3-8794-e53924379d25),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape3161de1-76') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG os_vif [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:71:14:95,bridge_name='br-int',has_traffic_filtering=True,id=e3161de1-7643-4b1a-8a0c-27791fed8dbd,network=Network(07d0f97b-06f9-40b3-8794-e53924379d25),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape3161de1-76') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tape3161de1-76, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tape3161de1-76, col_values=(('external_ids', {'iface-id': 'e3161de1-7643-4b1a-8a0c-27791fed8dbd', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:71:14:95', 'vm-uuid': '3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:17 user nova-compute[72081]: INFO os_vif [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:71:14:95,bridge_name='br-int',has_traffic_filtering=True,id=e3161de1-7643-4b1a-8a0c-27791fed8dbd,network=Network(07d0f97b-06f9-40b3-8794-e53924379d25),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape3161de1-76')
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:34:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] No VIF found with MAC fa:16:3e:71:14:95, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:34:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:18 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270458.3356135, b23d4bcd-989a-45e1-89a3-6e3d30229a32 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:34:18 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] VM Resumed (Lifecycle Event)
Apr 12 03:34:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:34:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:34:18 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Instance spawned successfully.
Apr 12 03:34:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:34:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: shelved_offloaded, current task_state: spawning, current DB power_state: 4, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:34:18 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:34:18 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270458.3360698, b23d4bcd-989a-45e1-89a3-6e3d30229a32 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:34:18 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] VM Started (Lifecycle Event)
Apr 12 03:34:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:34:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:34:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:34:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Synchronizing instance power state after lifecycle event "Started"; current vm_state: shelved_offloaded, current task_state: spawning, current DB power_state: 4, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:34:18 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:34:18 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270458.499971, 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:34:18 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] VM Resumed (Lifecycle Event)
Apr 12 03:34:18 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Instance spawned successfully.
Apr 12 03:34:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:34:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:34:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:34:18 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:34:18 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270458.5000803, 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:34:18 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] VM Started (Lifecycle Event)
Apr 12 03:34:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:34:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:34:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:34:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:34:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:34:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:34:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:34:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:34:18 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:34:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:34:18 user nova-compute[72081]: INFO nova.compute.manager [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Took 10.55 seconds to spawn the instance on the hypervisor.
Apr 12 03:34:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:34:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4434d048-8f03-4369-af7e-659bfde9cd73 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32" "released" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: held 21.572s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:18 user nova-compute[72081]: INFO nova.compute.manager [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Took 11.63 seconds to build instance.
Apr 12 03:34:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d786b7e7-1208-4ea7-bc41-3277b8e45960 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 11.820s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:19 user nova-compute[72081]: DEBUG nova.compute.manager [req-cacc92e0-b70f-46cc-926e-5103ff3839e2 req-5e3b5e65-b341-49ac-b6e9-d0e0ef59ea24 service nova] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Received event network-changed-e3161de1-7643-4b1a-8a0c-27791fed8dbd {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:34:19 user nova-compute[72081]: DEBUG nova.compute.manager [req-cacc92e0-b70f-46cc-926e-5103ff3839e2 req-5e3b5e65-b341-49ac-b6e9-d0e0ef59ea24 service nova] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Refreshing instance network info cache due to event network-changed-e3161de1-7643-4b1a-8a0c-27791fed8dbd. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:34:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cacc92e0-b70f-46cc-926e-5103ff3839e2 req-5e3b5e65-b341-49ac-b6e9-d0e0ef59ea24 service nova] Acquiring lock "refresh_cache-3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:34:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cacc92e0-b70f-46cc-926e-5103ff3839e2 req-5e3b5e65-b341-49ac-b6e9-d0e0ef59ea24 service nova] Acquired lock "refresh_cache-3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:34:19 user nova-compute[72081]: DEBUG nova.network.neutron [req-cacc92e0-b70f-46cc-926e-5103ff3839e2 req-5e3b5e65-b341-49ac-b6e9-d0e0ef59ea24 service nova] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Refreshing network info cache for port e3161de1-7643-4b1a-8a0c-27791fed8dbd {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:34:19 user nova-compute[72081]: DEBUG nova.network.neutron [req-cacc92e0-b70f-46cc-926e-5103ff3839e2 req-5e3b5e65-b341-49ac-b6e9-d0e0ef59ea24 service nova] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Updated VIF entry in instance network info cache for port e3161de1-7643-4b1a-8a0c-27791fed8dbd. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:34:19 user nova-compute[72081]: DEBUG nova.network.neutron [req-cacc92e0-b70f-46cc-926e-5103ff3839e2 req-5e3b5e65-b341-49ac-b6e9-d0e0ef59ea24 service nova] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Updating instance_info_cache with network_info: [{"id": "e3161de1-7643-4b1a-8a0c-27791fed8dbd", "address": "fa:16:3e:71:14:95", "network": {"id": "07d0f97b-06f9-40b3-8794-e53924379d25", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-2076424457-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "86f7b42b161f41efb234f0ce70210a08", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3161de1-76", "ovs_interfaceid": "e3161de1-7643-4b1a-8a0c-27791fed8dbd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:34:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cacc92e0-b70f-46cc-926e-5103ff3839e2 req-5e3b5e65-b341-49ac-b6e9-d0e0ef59ea24 service nova] Releasing lock "refresh_cache-3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:34:19 user nova-compute[72081]: DEBUG nova.compute.manager [req-cacc92e0-b70f-46cc-926e-5103ff3839e2 req-5e3b5e65-b341-49ac-b6e9-d0e0ef59ea24 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received event network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:34:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cacc92e0-b70f-46cc-926e-5103ff3839e2 req-5e3b5e65-b341-49ac-b6e9-d0e0ef59ea24 service nova] Acquiring lock "72ba1218-065d-40c4-8503-7c4155e47503-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cacc92e0-b70f-46cc-926e-5103ff3839e2 req-5e3b5e65-b341-49ac-b6e9-d0e0ef59ea24 service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cacc92e0-b70f-46cc-926e-5103ff3839e2 req-5e3b5e65-b341-49ac-b6e9-d0e0ef59ea24 service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:19 user nova-compute[72081]: DEBUG nova.compute.manager [req-cacc92e0-b70f-46cc-926e-5103ff3839e2 req-5e3b5e65-b341-49ac-b6e9-d0e0ef59ea24 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] No waiting events found dispatching network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:34:19 user nova-compute[72081]: WARNING nova.compute.manager [req-cacc92e0-b70f-46cc-926e-5103ff3839e2 req-5e3b5e65-b341-49ac-b6e9-d0e0ef59ea24 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received unexpected event network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 for instance with vm_state active and task_state rescuing.
Apr 12 03:34:19 user nova-compute[72081]: DEBUG nova.compute.manager [req-cacc92e0-b70f-46cc-926e-5103ff3839e2 req-5e3b5e65-b341-49ac-b6e9-d0e0ef59ea24 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received event network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:34:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cacc92e0-b70f-46cc-926e-5103ff3839e2 req-5e3b5e65-b341-49ac-b6e9-d0e0ef59ea24 service nova] Acquiring lock "72ba1218-065d-40c4-8503-7c4155e47503-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cacc92e0-b70f-46cc-926e-5103ff3839e2 req-5e3b5e65-b341-49ac-b6e9-d0e0ef59ea24 service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cacc92e0-b70f-46cc-926e-5103ff3839e2 req-5e3b5e65-b341-49ac-b6e9-d0e0ef59ea24 service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:19 user nova-compute[72081]: DEBUG nova.compute.manager [req-cacc92e0-b70f-46cc-926e-5103ff3839e2 req-5e3b5e65-b341-49ac-b6e9-d0e0ef59ea24 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] No waiting events found dispatching network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:34:19 user nova-compute[72081]: WARNING nova.compute.manager [req-cacc92e0-b70f-46cc-926e-5103ff3839e2 req-5e3b5e65-b341-49ac-b6e9-d0e0ef59ea24 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received unexpected event network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 for instance with vm_state active and task_state rescuing.
Apr 12 03:34:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:20 user nova-compute[72081]: DEBUG nova.compute.manager [req-4c3ac44e-e0c3-4b48-b24e-b07ca6d748f6 req-d537b203-63a1-4709-9f45-5b2e03ad453c service nova] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Received event network-vif-plugged-e3161de1-7643-4b1a-8a0c-27791fed8dbd {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:34:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4c3ac44e-e0c3-4b48-b24e-b07ca6d748f6 req-d537b203-63a1-4709-9f45-5b2e03ad453c service nova] Acquiring lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4c3ac44e-e0c3-4b48-b24e-b07ca6d748f6 req-d537b203-63a1-4709-9f45-5b2e03ad453c service nova] Lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4c3ac44e-e0c3-4b48-b24e-b07ca6d748f6 req-d537b203-63a1-4709-9f45-5b2e03ad453c service nova] Lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:20 user nova-compute[72081]: DEBUG nova.compute.manager [req-4c3ac44e-e0c3-4b48-b24e-b07ca6d748f6 req-d537b203-63a1-4709-9f45-5b2e03ad453c service nova] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] No waiting events found dispatching network-vif-plugged-e3161de1-7643-4b1a-8a0c-27791fed8dbd {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:34:20 user nova-compute[72081]: WARNING nova.compute.manager [req-4c3ac44e-e0c3-4b48-b24e-b07ca6d748f6 req-d537b203-63a1-4709-9f45-5b2e03ad453c service nova] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Received unexpected event network-vif-plugged-e3161de1-7643-4b1a-8a0c-27791fed8dbd for instance with vm_state building and task_state spawning.
Apr 12 03:34:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:22 user nova-compute[72081]: DEBUG nova.compute.manager [req-f8db0848-12fe-44d8-b4d5-79a1489f38bb req-946804b0-aece-481e-8ee1-331f8591dd7a service nova] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Received event network-vif-plugged-e3161de1-7643-4b1a-8a0c-27791fed8dbd {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:34:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f8db0848-12fe-44d8-b4d5-79a1489f38bb req-946804b0-aece-481e-8ee1-331f8591dd7a service nova] Acquiring lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f8db0848-12fe-44d8-b4d5-79a1489f38bb req-946804b0-aece-481e-8ee1-331f8591dd7a service nova] Lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f8db0848-12fe-44d8-b4d5-79a1489f38bb req-946804b0-aece-481e-8ee1-331f8591dd7a service nova] Lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:22 user nova-compute[72081]: DEBUG nova.compute.manager [req-f8db0848-12fe-44d8-b4d5-79a1489f38bb req-946804b0-aece-481e-8ee1-331f8591dd7a service nova] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] No waiting events found dispatching network-vif-plugged-e3161de1-7643-4b1a-8a0c-27791fed8dbd {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:34:22 user nova-compute[72081]: WARNING nova.compute.manager [req-f8db0848-12fe-44d8-b4d5-79a1489f38bb req-946804b0-aece-481e-8ee1-331f8591dd7a service nova] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Received unexpected event network-vif-plugged-e3161de1-7643-4b1a-8a0c-27791fed8dbd for instance with vm_state building and task_state spawning.
Apr 12 03:34:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Removed pending event for 72ba1218-065d-40c4-8503-7c4155e47503 due to event {{(pid=72081) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 12 03:34:22 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270462.6215613, 72ba1218-065d-40c4-8503-7c4155e47503 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:34:22 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] VM Resumed (Lifecycle Event)
Apr 12 03:34:22 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4014d9d6-b6e2-45bc-bb8c-d07dece28fb9 tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:34:22 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:34:22 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:34:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:22 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] During sync_power_state the instance has a pending task (rescuing). Skip.
Apr 12 03:34:22 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270462.6232834, 72ba1218-065d-40c4-8503-7c4155e47503 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:34:22 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] VM Started (Lifecycle Event)
Apr 12 03:34:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:22 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:34:22 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Synchronizing instance power state after lifecycle event "Started"; current vm_state: rescued, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:34:23 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270463.0989456, 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:34:23 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] VM Resumed (Lifecycle Event)
Apr 12 03:34:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:34:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:34:23 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Instance spawned successfully.
Apr 12 03:34:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:34:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:34:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:34:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:34:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:34:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:34:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:34:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:34:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:34:23 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:34:23 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270463.1034951, 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:34:23 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] VM Started (Lifecycle Event)
Apr 12 03:34:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:34:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:34:23 user nova-compute[72081]: INFO nova.compute.manager [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Took 9.59 seconds to spawn the instance on the hypervisor.
Apr 12 03:34:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:34:23 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:34:23 user nova-compute[72081]: INFO nova.compute.manager [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Took 10.51 seconds to build instance.
Apr 12 03:34:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bba1a3f9-6667-4ec1-88e2-d95fecfb4c8d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 10.619s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "23367b24-0a3b-454d-901d-8a3e5396333a" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "23367b24-0a3b-454d-901d-8a3e5396333a" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:30 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:34:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:30 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:34:30 user nova-compute[72081]: INFO nova.compute.claims [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Claim successful on node user
Apr 12 03:34:31 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:34:31 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:34:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.516s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:31 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:34:31 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:34:31 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:34:31 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:34:31 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:34:31 user nova-compute[72081]: DEBUG nova.policy [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '3f8024260bf14d9995e67cb0d1bc8ddc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05e1e25933cd42df98fb060b8d2701dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:34:31 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:34:31 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:34:31 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Creating image(s)
Apr 12 03:34:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "/opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "/opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "/opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.008s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "20a9ef7c61ef4dc87afe749fc5ad5499efda7617" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "20a9ef7c61ef4dc87afe749fc5ad5499efda7617" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/20a9ef7c61ef4dc87afe749fc5ad5499efda7617.part --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/20a9ef7c61ef4dc87afe749fc5ad5499efda7617.part --force-share --output=json" returned: 0 in 0.334s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:32 user nova-compute[72081]: DEBUG nova.virt.images [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] 5267358d-2283-43f2-a893-8842d5de70da was qcow2, converting to raw {{(pid=72081) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 12 03:34:32 user nova-compute[72081]: DEBUG nova.privsep.utils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72081) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 12 03:34:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/20a9ef7c61ef4dc87afe749fc5ad5499efda7617.part /opt/stack/data/nova/instances/_base/20a9ef7c61ef4dc87afe749fc5ad5499efda7617.converted {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:32 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Successfully created port: a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:34:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/20a9ef7c61ef4dc87afe749fc5ad5499efda7617.part /opt/stack/data/nova/instances/_base/20a9ef7c61ef4dc87afe749fc5ad5499efda7617.converted" returned: 0 in 0.302s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/20a9ef7c61ef4dc87afe749fc5ad5499efda7617.converted --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/20a9ef7c61ef4dc87afe749fc5ad5499efda7617.converted --force-share --output=json" returned: 0 in 0.214s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "20a9ef7c61ef4dc87afe749fc5ad5499efda7617" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.288s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/20a9ef7c61ef4dc87afe749fc5ad5499efda7617 --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/20a9ef7c61ef4dc87afe749fc5ad5499efda7617 --force-share --output=json" returned: 0 in 0.162s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "20a9ef7c61ef4dc87afe749fc5ad5499efda7617" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "20a9ef7c61ef4dc87afe749fc5ad5499efda7617" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/20a9ef7c61ef4dc87afe749fc5ad5499efda7617 --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/20a9ef7c61ef4dc87afe749fc5ad5499efda7617 --force-share --output=json" returned: 0 in 0.202s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/20a9ef7c61ef4dc87afe749fc5ad5499efda7617,backing_fmt=raw /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/20a9ef7c61ef4dc87afe749fc5ad5499efda7617,backing_fmt=raw /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk 1073741824" returned: 0 in 0.062s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "20a9ef7c61ef4dc87afe749fc5ad5499efda7617" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.281s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/20a9ef7c61ef4dc87afe749fc5ad5499efda7617 --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Successfully updated port: a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/20a9ef7c61ef4dc87afe749fc5ad5499efda7617 --force-share --output=json" returned: 0 in 0.146s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Checking if we can resize image /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-ba23212f-e79e-45fe-b670-0beaf33c641a req-f0a0849a-5f8a-4bf2-a771-7c40a1a2188e service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Received event network-changed-a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-ba23212f-e79e-45fe-b670-0beaf33c641a req-f0a0849a-5f8a-4bf2-a771-7c40a1a2188e service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Refreshing instance network info cache due to event network-changed-a76c6f07-234b-4832-8ace-ec81656b06a7. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ba23212f-e79e-45fe-b670-0beaf33c641a req-f0a0849a-5f8a-4bf2-a771-7c40a1a2188e service nova] Acquiring lock "refresh_cache-23367b24-0a3b-454d-901d-8a3e5396333a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ba23212f-e79e-45fe-b670-0beaf33c641a req-f0a0849a-5f8a-4bf2-a771-7c40a1a2188e service nova] Acquired lock "refresh_cache-23367b24-0a3b-454d-901d-8a3e5396333a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG nova.network.neutron [req-ba23212f-e79e-45fe-b670-0beaf33c641a req-f0a0849a-5f8a-4bf2-a771-7c40a1a2188e service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Refreshing network info cache for port a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "refresh_cache-23367b24-0a3b-454d-901d-8a3e5396333a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG nova.network.neutron [req-ba23212f-e79e-45fe-b670-0beaf33c641a req-f0a0849a-5f8a-4bf2-a771-7c40a1a2188e service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json" returned: 0 in 0.318s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Cannot resize image /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG nova.objects.instance [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lazy-loading 'migration_context' on Instance uuid 23367b24-0a3b-454d-901d-8a3e5396333a {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Ensure instance console log exists: /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG nova.network.neutron [req-ba23212f-e79e-45fe-b670-0beaf33c641a req-f0a0849a-5f8a-4bf2-a771-7c40a1a2188e service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ba23212f-e79e-45fe-b670-0beaf33c641a req-f0a0849a-5f8a-4bf2-a771-7c40a1a2188e service nova] Releasing lock "refresh_cache-23367b24-0a3b-454d-901d-8a3e5396333a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquired lock "refresh_cache-23367b24-0a3b-454d-901d-8a3e5396333a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:34:33 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Updating instance_info_cache with network_info: [{"id": "a76c6f07-234b-4832-8ace-ec81656b06a7", "address": "fa:16:3e:e4:18:3e", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa76c6f07-23", "ovs_interfaceid": "a76c6f07-234b-4832-8ace-ec81656b06a7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Releasing lock "refresh_cache-23367b24-0a3b-454d-901d-8a3e5396333a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Instance network_info: |[{"id": "a76c6f07-234b-4832-8ace-ec81656b06a7", "address": "fa:16:3e:e4:18:3e", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa76c6f07-23", "ovs_interfaceid": "a76c6f07-234b-4832-8ace-ec81656b06a7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Start _get_guest_xml network_info=[{"id": "a76c6f07-234b-4832-8ace-ec81656b06a7", "address": "fa:16:3e:e4:18:3e", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa76c6f07-23", "ovs_interfaceid": "a76c6f07-234b-4832-8ace-ec81656b06a7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:34:27Z,direct_url=<?>,disk_format='qcow2',id=5267358d-2283-43f2-a893-8842d5de70da,min_disk=0,min_ram=0,name='tempest-scenario-img--564335850',owner='05e1e25933cd42df98fb060b8d2701dc',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:34:29Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': '5267358d-2283-43f2-a893-8842d5de70da'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:34:34 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:34:34 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:34:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:34:27Z,direct_url=<?>,disk_format='qcow2',id=5267358d-2283-43f2-a893-8842d5de70da,min_disk=0,min_ram=0,name='tempest-scenario-img--564335850',owner='05e1e25933cd42df98fb060b8d2701dc',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:34:29Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:34:29Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-2135071440',display_name='tempest-TestMinimumBasicScenario-server-2135071440',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-2135071440',id=17,image_ref='5267358d-2283-43f2-a893-8842d5de70da',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMkexWkk4Rxu6k02iex2AsbJ+PDY5htmdDN8vQSba2S3zwrjAPkO8wlI5tbs0OLJWNWhrMe6/AoKb3eoA7zRDJCJlh8REZk8FLy3CbfuwbBsBPrnYnRJ2rfKKtaAGF256w==',key_name='tempest-TestMinimumBasicScenario-2058975683',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='05e1e25933cd42df98fb060b8d2701dc',ramdisk_id='',reservation_id='r-0n9gnxqj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='5267358d-2283-43f2-a893-8842d5de70da',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-TestMinimumBasicScenario-1605347971',owner_user_name='tempest-TestMinimumBasicScenario-1605347971-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:34:31Z,user_data=None,user_id='3f8024260bf14d9995e67cb0d1bc8ddc',uuid=23367b24-0a3b-454d-901d-8a3e5396333a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a76c6f07-234b-4832-8ace-ec81656b06a7", "address": "fa:16:3e:e4:18:3e", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa76c6f07-23", "ovs_interfaceid": "a76c6f07-234b-4832-8ace-ec81656b06a7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Converting VIF {"id": "a76c6f07-234b-4832-8ace-ec81656b06a7", "address": "fa:16:3e:e4:18:3e", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa76c6f07-23", "ovs_interfaceid": "a76c6f07-234b-4832-8ace-ec81656b06a7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e4:18:3e,bridge_name='br-int',has_traffic_filtering=True,id=a76c6f07-234b-4832-8ace-ec81656b06a7,network=Network(91a7501a-44a5-4d48-a1f7-e1195ac99607),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa76c6f07-23') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG nova.objects.instance [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lazy-loading 'pci_devices' on Instance uuid 23367b24-0a3b-454d-901d-8a3e5396333a {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:34:34 user nova-compute[72081]:   <uuid>23367b24-0a3b-454d-901d-8a3e5396333a</uuid>
Apr 12 03:34:34 user nova-compute[72081]:   <name>instance-00000011</name>
Apr 12 03:34:34 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:34:34 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:34:34 user nova-compute[72081]:   <metadata>
Apr 12 03:34:34 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:34:34 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:34:34 user nova-compute[72081]:       <nova:name>tempest-TestMinimumBasicScenario-server-2135071440</nova:name>
Apr 12 03:34:34 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:34:34</nova:creationTime>
Apr 12 03:34:34 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:34:34 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:34:34 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:34:34 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:34:34 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:34:34 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:34:34 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:34:34 user nova-compute[72081]:       <nova:owner>
Apr 12 03:34:34 user nova-compute[72081]:         <nova:user uuid="3f8024260bf14d9995e67cb0d1bc8ddc">tempest-TestMinimumBasicScenario-1605347971-project-member</nova:user>
Apr 12 03:34:34 user nova-compute[72081]:         <nova:project uuid="05e1e25933cd42df98fb060b8d2701dc">tempest-TestMinimumBasicScenario-1605347971</nova:project>
Apr 12 03:34:34 user nova-compute[72081]:       </nova:owner>
Apr 12 03:34:34 user nova-compute[72081]:       <nova:root type="image" uuid="5267358d-2283-43f2-a893-8842d5de70da"/>
Apr 12 03:34:34 user nova-compute[72081]:       <nova:ports>
Apr 12 03:34:34 user nova-compute[72081]:         <nova:port uuid="a76c6f07-234b-4832-8ace-ec81656b06a7">
Apr 12 03:34:34 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 12 03:34:34 user nova-compute[72081]:         </nova:port>
Apr 12 03:34:34 user nova-compute[72081]:       </nova:ports>
Apr 12 03:34:34 user nova-compute[72081]:     </nova:instance>
Apr 12 03:34:34 user nova-compute[72081]:   </metadata>
Apr 12 03:34:34 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:34:34 user nova-compute[72081]:     <system>
Apr 12 03:34:34 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:34:34 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:34:34 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:34:34 user nova-compute[72081]:       <entry name="serial">23367b24-0a3b-454d-901d-8a3e5396333a</entry>
Apr 12 03:34:34 user nova-compute[72081]:       <entry name="uuid">23367b24-0a3b-454d-901d-8a3e5396333a</entry>
Apr 12 03:34:34 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:34:34 user nova-compute[72081]:     </system>
Apr 12 03:34:34 user nova-compute[72081]:   </sysinfo>
Apr 12 03:34:34 user nova-compute[72081]:   <os>
Apr 12 03:34:34 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:34:34 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:34:34 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:34:34 user nova-compute[72081]:   </os>
Apr 12 03:34:34 user nova-compute[72081]:   <features>
Apr 12 03:34:34 user nova-compute[72081]:     <acpi/>
Apr 12 03:34:34 user nova-compute[72081]:     <apic/>
Apr 12 03:34:34 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:34:34 user nova-compute[72081]:   </features>
Apr 12 03:34:34 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:34:34 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:34:34 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:34:34 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:34:34 user nova-compute[72081]:   </clock>
Apr 12 03:34:34 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:34:34 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:34:34 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:34:34 user nova-compute[72081]:   </cpu>
Apr 12 03:34:34 user nova-compute[72081]:   <devices>
Apr 12 03:34:34 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:34:34 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:34:34 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk"/>
Apr 12 03:34:34 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:34:34 user nova-compute[72081]:     </disk>
Apr 12 03:34:34 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:34:34 user nova-compute[72081]:       <mac address="fa:16:3e:e4:18:3e"/>
Apr 12 03:34:34 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:34:34 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:34:34 user nova-compute[72081]:       <target dev="tapa76c6f07-23"/>
Apr 12 03:34:34 user nova-compute[72081]:     </interface>
Apr 12 03:34:34 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:34:34 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/console.log" append="off"/>
Apr 12 03:34:34 user nova-compute[72081]:     </serial>
Apr 12 03:34:34 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:34:34 user nova-compute[72081]:     <video>
Apr 12 03:34:34 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:34:34 user nova-compute[72081]:     </video>
Apr 12 03:34:34 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:34:34 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:34:34 user nova-compute[72081]:     </rng>
Apr 12 03:34:34 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:34:34 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:34:34 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:34:34 user nova-compute[72081]:     </memballoon>
Apr 12 03:34:34 user nova-compute[72081]:   </devices>
Apr 12 03:34:34 user nova-compute[72081]: </domain>
Apr 12 03:34:34 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:34:29Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-2135071440',display_name='tempest-TestMinimumBasicScenario-server-2135071440',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-2135071440',id=17,image_ref='5267358d-2283-43f2-a893-8842d5de70da',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMkexWkk4Rxu6k02iex2AsbJ+PDY5htmdDN8vQSba2S3zwrjAPkO8wlI5tbs0OLJWNWhrMe6/AoKb3eoA7zRDJCJlh8REZk8FLy3CbfuwbBsBPrnYnRJ2rfKKtaAGF256w==',key_name='tempest-TestMinimumBasicScenario-2058975683',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='05e1e25933cd42df98fb060b8d2701dc',ramdisk_id='',reservation_id='r-0n9gnxqj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='5267358d-2283-43f2-a893-8842d5de70da',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-TestMinimumBasicScenario-1605347971',owner_user_name='tempest-TestMinimumBasicScenario-1605347971-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:34:31Z,user_data=None,user_id='3f8024260bf14d9995e67cb0d1bc8ddc',uuid=23367b24-0a3b-454d-901d-8a3e5396333a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a76c6f07-234b-4832-8ace-ec81656b06a7", "address": "fa:16:3e:e4:18:3e", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa76c6f07-23", "ovs_interfaceid": "a76c6f07-234b-4832-8ace-ec81656b06a7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Converting VIF {"id": "a76c6f07-234b-4832-8ace-ec81656b06a7", "address": "fa:16:3e:e4:18:3e", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa76c6f07-23", "ovs_interfaceid": "a76c6f07-234b-4832-8ace-ec81656b06a7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e4:18:3e,bridge_name='br-int',has_traffic_filtering=True,id=a76c6f07-234b-4832-8ace-ec81656b06a7,network=Network(91a7501a-44a5-4d48-a1f7-e1195ac99607),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa76c6f07-23') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG os_vif [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:e4:18:3e,bridge_name='br-int',has_traffic_filtering=True,id=a76c6f07-234b-4832-8ace-ec81656b06a7,network=Network(91a7501a-44a5-4d48-a1f7-e1195ac99607),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa76c6f07-23') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapa76c6f07-23, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapa76c6f07-23, col_values=(('external_ids', {'iface-id': 'a76c6f07-234b-4832-8ace-ec81656b06a7', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:e4:18:3e', 'vm-uuid': '23367b24-0a3b-454d-901d-8a3e5396333a'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:34 user nova-compute[72081]: INFO os_vif [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:e4:18:3e,bridge_name='br-int',has_traffic_filtering=True,id=a76c6f07-234b-4832-8ace-ec81656b06a7,network=Network(91a7501a-44a5-4d48-a1f7-e1195ac99607),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa76c6f07-23')
Apr 12 03:34:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:34:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] No VIF found with MAC fa:16:3e:e4:18:3e, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:34:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:36 user nova-compute[72081]: DEBUG nova.compute.manager [req-206739e5-bc7d-4df5-9340-cfe3706ad427 req-fcb1736b-7027-4d95-af7b-f44da024880a service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Received event network-vif-plugged-a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:34:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-206739e5-bc7d-4df5-9340-cfe3706ad427 req-fcb1736b-7027-4d95-af7b-f44da024880a service nova] Acquiring lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-206739e5-bc7d-4df5-9340-cfe3706ad427 req-fcb1736b-7027-4d95-af7b-f44da024880a service nova] Lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-206739e5-bc7d-4df5-9340-cfe3706ad427 req-fcb1736b-7027-4d95-af7b-f44da024880a service nova] Lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:36 user nova-compute[72081]: DEBUG nova.compute.manager [req-206739e5-bc7d-4df5-9340-cfe3706ad427 req-fcb1736b-7027-4d95-af7b-f44da024880a service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] No waiting events found dispatching network-vif-plugged-a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:34:36 user nova-compute[72081]: WARNING nova.compute.manager [req-206739e5-bc7d-4df5-9340-cfe3706ad427 req-fcb1736b-7027-4d95-af7b-f44da024880a service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Received unexpected event network-vif-plugged-a76c6f07-234b-4832-8ace-ec81656b06a7 for instance with vm_state building and task_state spawning.
Apr 12 03:34:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-f3a39ab6-f287-4c0d-b814-b5364ddb75c1 req-9ba584d7-b588-4e3c-a319-50bc4da4f576 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Received event network-vif-plugged-a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:34:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f3a39ab6-f287-4c0d-b814-b5364ddb75c1 req-9ba584d7-b588-4e3c-a319-50bc4da4f576 service nova] Acquiring lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:34:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f3a39ab6-f287-4c0d-b814-b5364ddb75c1 req-9ba584d7-b588-4e3c-a319-50bc4da4f576 service nova] Lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:34:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f3a39ab6-f287-4c0d-b814-b5364ddb75c1 req-9ba584d7-b588-4e3c-a319-50bc4da4f576 service nova] Lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-f3a39ab6-f287-4c0d-b814-b5364ddb75c1 req-9ba584d7-b588-4e3c-a319-50bc4da4f576 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] No waiting events found dispatching network-vif-plugged-a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:34:38 user nova-compute[72081]: WARNING nova.compute.manager [req-f3a39ab6-f287-4c0d-b814-b5364ddb75c1 req-9ba584d7-b588-4e3c-a319-50bc4da4f576 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Received unexpected event network-vif-plugged-a76c6f07-234b-4832-8ace-ec81656b06a7 for instance with vm_state building and task_state spawning.
Apr 12 03:34:38 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270478.6334186, 23367b24-0a3b-454d-901d-8a3e5396333a => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:34:38 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] VM Resumed (Lifecycle Event)
Apr 12 03:34:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:34:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:34:38 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Instance spawned successfully.
Apr 12 03:34:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:34:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:34:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:34:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:34:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:34:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:34:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:34:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:34:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:34:38 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:34:38 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270478.6382108, 23367b24-0a3b-454d-901d-8a3e5396333a => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:34:38 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] VM Started (Lifecycle Event)
Apr 12 03:34:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:34:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:34:38 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:34:38 user nova-compute[72081]: INFO nova.compute.manager [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Took 7.35 seconds to spawn the instance on the hypervisor.
Apr 12 03:34:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:34:38 user nova-compute[72081]: INFO nova.compute.manager [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Took 8.31 seconds to build instance.
Apr 12 03:34:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ec4a6ff-c816-4358-9aca-dde9ce1da36a tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "23367b24-0a3b-454d-901d-8a3e5396333a" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 8.427s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:34:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:57 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:34:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:58 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:34:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:34:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:00 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:35:00 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:35:00 user nova-compute[72081]: INFO nova.compute.manager [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Rescuing
Apr 12 03:35:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "refresh_cache-bcd267d2-5b2c-4ed2-b634-0210fae94dfd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:35:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquired lock "refresh_cache-bcd267d2-5b2c-4ed2-b634-0210fae94dfd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:35:00 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:35:01 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:35:01 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:35:01 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:35:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:35:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:35:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:35:01 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:35:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:01 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Updating instance_info_cache with network_info: [{"id": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "address": "fa:16:3e:9d:0c:c2", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa4385731-05", "ovs_interfaceid": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:35:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Releasing lock "refresh_cache-bcd267d2-5b2c-4ed2-b634-0210fae94dfd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:35:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.189s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:02 user nova-compute[72081]: DEBUG nova.compute.manager [req-9f105538-0ce4-4c49-adfb-253c302eea5b req-a667ed34-1c00-4d7c-8f3c-9a0acbcee8e5 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received event network-vif-unplugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:35:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9f105538-0ce4-4c49-adfb-253c302eea5b req-a667ed34-1c00-4d7c-8f3c-9a0acbcee8e5 service nova] Acquiring lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:35:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9f105538-0ce4-4c49-adfb-253c302eea5b req-a667ed34-1c00-4d7c-8f3c-9a0acbcee8e5 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:35:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9f105538-0ce4-4c49-adfb-253c302eea5b req-a667ed34-1c00-4d7c-8f3c-9a0acbcee8e5 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:35:02 user nova-compute[72081]: DEBUG nova.compute.manager [req-9f105538-0ce4-4c49-adfb-253c302eea5b req-a667ed34-1c00-4d7c-8f3c-9a0acbcee8e5 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] No waiting events found dispatching network-vif-unplugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:35:02 user nova-compute[72081]: WARNING nova.compute.manager [req-9f105538-0ce4-4c49-adfb-253c302eea5b req-a667ed34-1c00-4d7c-8f3c-9a0acbcee8e5 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received unexpected event network-vif-unplugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 for instance with vm_state active and task_state rescuing.
Apr 12 03:35:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk --force-share --output=json" returned: 0 in 0.156s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:02 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdd (vdc) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:35:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json" returned: 0 in 0.177s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json" returned: 0 in 0.181s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:02 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Instance destroyed successfully.
Apr 12 03:35:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:02 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Attempting a stable device rescue
Apr 12 03:35:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} {{(pid=72081) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4289}}
Apr 12 03:35:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Instance directory exists: not creating {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4694}}
Apr 12 03:35:03 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Creating image(s)
Apr 12 03:35:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "/opt/stack/data/nova/instances/bcd267d2-5b2c-4ed2-b634-0210fae94dfd/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:35:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "/opt/stack/data/nova/instances/bcd267d2-5b2c-4ed2-b634-0210fae94dfd/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:35:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "/opt/stack/data/nova/instances/bcd267d2-5b2c-4ed2-b634-0210fae94dfd/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.005s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:35:03 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lazy-loading 'trusted_certs' on Instance uuid bcd267d2-5b2c-4ed2-b634-0210fae94dfd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:35:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.158s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "97536260b2fcf56105eaaf8a9865fca3acbfc0a0" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:35:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "97536260b2fcf56105eaaf8a9865fca3acbfc0a0" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.003s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:35:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/97536260b2fcf56105eaaf8a9865fca3acbfc0a0.part --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json" returned: 0 in 0.934s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/97536260b2fcf56105eaaf8a9865fca3acbfc0a0.part --force-share --output=json" returned: 0 in 0.169s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG nova.virt.images [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] 26f3e3a3-e498-43b6-bced-72fd4d993267 was qcow2, converting to raw {{(pid=72081) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG nova.privsep.utils [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72081) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/97536260b2fcf56105eaaf8a9865fca3acbfc0a0.part /opt/stack/data/nova/instances/_base/97536260b2fcf56105eaaf8a9865fca3acbfc0a0.converted {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk.rescue --force-share --output=json" returned: 0 in 0.133s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-6bad47d5-1bd8-4201-b143-225ef8a434b3 req-5b7e0f23-5994-4e99-ba37-0d8360eaf18f service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6bad47d5-1bd8-4201-b143-225ef8a434b3 req-5b7e0f23-5994-4e99-ba37-0d8360eaf18f service nova] Acquiring lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6bad47d5-1bd8-4201-b143-225ef8a434b3 req-5b7e0f23-5994-4e99-ba37-0d8360eaf18f service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6bad47d5-1bd8-4201-b143-225ef8a434b3 req-5b7e0f23-5994-4e99-ba37-0d8360eaf18f service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-6bad47d5-1bd8-4201-b143-225ef8a434b3 req-5b7e0f23-5994-4e99-ba37-0d8360eaf18f service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] No waiting events found dispatching network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:35:04 user nova-compute[72081]: WARNING nova.compute.manager [req-6bad47d5-1bd8-4201-b143-225ef8a434b3 req-5b7e0f23-5994-4e99-ba37-0d8360eaf18f service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received unexpected event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 for instance with vm_state active and task_state rescuing.
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk.rescue --force-share --output=json" returned: 0 in 0.142s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/97536260b2fcf56105eaaf8a9865fca3acbfc0a0.part /opt/stack/data/nova/instances/_base/97536260b2fcf56105eaaf8a9865fca3acbfc0a0.converted" returned: 0 in 0.383s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/97536260b2fcf56105eaaf8a9865fca3acbfc0a0.converted --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk --force-share --output=json" returned: 0 in 0.162s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/97536260b2fcf56105eaaf8a9865fca3acbfc0a0.converted --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "97536260b2fcf56105eaaf8a9865fca3acbfc0a0" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.700s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "97536260b2fcf56105eaaf8a9865fca3acbfc0a0" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "97536260b2fcf56105eaaf8a9865fca3acbfc0a0" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/97536260b2fcf56105eaaf8a9865fca3acbfc0a0 --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/97536260b2fcf56105eaaf8a9865fca3acbfc0a0 --force-share --output=json" returned: 0 in 0.156s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/97536260b2fcf56105eaaf8a9865fca3acbfc0a0,backing_fmt=raw /opt/stack/data/nova/instances/bcd267d2-5b2c-4ed2-b634-0210fae94dfd/disk.rescue {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/97536260b2fcf56105eaaf8a9865fca3acbfc0a0,backing_fmt=raw /opt/stack/data/nova/instances/bcd267d2-5b2c-4ed2-b634-0210fae94dfd/disk.rescue" returned: 0 in 0.061s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "97536260b2fcf56105eaaf8a9865fca3acbfc0a0" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.226s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lazy-loading 'migration_context' on Instance uuid bcd267d2-5b2c-4ed2-b634-0210fae94dfd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Start _get_guest_xml network_info=[{"id": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "address": "fa:16:3e:9d:0c:c2", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "vif_mac": "fa:16:3e:9d:0c:c2"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa4385731-05", "ovs_interfaceid": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': '26f3e3a3-e498-43b6-bced-72fd4d993267', 'kernel_id': '', 'ramdisk_id': ''} block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FfmHBZWX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, 'status': 'reserved', 'instance': 'bcd267d2-5b2c-4ed2-b634-0210fae94dfd', 'attached_at': '', 'detached_at': '', 'volume_id': 'c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2', 'serial': 'c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2'}, 'disk_bus': 'virtio', 'delete_on_termination': False, 'boot_index': 0, 'guest_format': None, 'device_type': 'disk', 'attachment_id': 'f3dac36a-808b-4e7a-84dd-5079aba3a56e', 'volume_type': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lazy-loading 'resources' on Instance uuid bcd267d2-5b2c-4ed2-b634-0210fae94dfd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lazy-loading 'numa_topology' on Instance uuid bcd267d2-5b2c-4ed2-b634-0210fae94dfd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:35:05 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:35:05 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:35:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lazy-loading 'vcpu_model' on Instance uuid bcd267d2-5b2c-4ed2-b634-0210fae94dfd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FfmHBZWX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:35:05 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:35:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:35:05 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:35:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:35:05 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[06be15d7-627d-4465-a550-cb0a6f20415a]: (4, ('# BEGIN RECORD 2.1.5\nnode.name = iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = <empty>\niface.ipaddress = <empty>\niface.prefix_len = 0\niface.hwaddress = <empty>\niface.transport_name = tcp\niface.initiatorname = <empty>\niface.state = <empty>\niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = <empty>\niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = <empty>\niface.subnet_mask = <empty>\niface.gateway = <empty>\niface.dhcp_alt_client_id_state = <empty>\niface.dhcp_alt_client_id = <empty>\niface.dhcp_dns = <empty>\niface.dhcp_learn_iqn = <empty>\niface.dhcp_req_vendor_id_state = <empty>\niface.dhcp_vendor_id_state = <empty>\niface.dhcp_vendor_id = <empty>\niface.dhcp_slp_da = <empty>\niface.fragmentation = <empty>\niface.gratuitous_arp = <empty>\niface.incoming_forwarding = <empty>\niface.tos_state = <empty>\niface.tos = 0\niface.ttl = 0\niface.delayed_ack = <empty>\niface.tcp_nagle = <empty>\niface.tcp_wsf_state = <empty>\niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = <empty>\niface.redirect = <empty>\niface.def_task_mgmt_timeout = 0\niface.header_digest = <empty>\niface.data_digest = <empty>\niface.immediate_data = <empty>\niface.initial_r2t = <empty>\niface.data_seq_inorder = <empty>\niface.data_pdu_inorder = <empty>\niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = <empty>\niface.bidi_chap = <empty>\niface.strict_login_compliance = <empty>\niface.discovery_auth = <empty>\niface.discovery_logout = <empty>\nnode.discovery_address = <empty>\nnode.discovery_port = 0\nnode.discovery_type = static\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = -20\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = CHAP\nnode.session.auth.username = FfmHBZWX\nnode.session.auth.password = ********\nnode.session.auth.username_in = <empty>\nnode.session.auth.password_in = <empty>\nnode.session.auth.chap_algs = MD5\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = manual\nnode.session.reopen_max = 0\nnode.conn[0].address = 172.16.0.220\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm (): stdout=# BEGIN RECORD 2.1.5
Apr 12 03:35:05 user nova-compute[72081]: node.name = iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:35:05 user nova-compute[72081]: node.tpgt = -1
Apr 12 03:35:05 user nova-compute[72081]: node.startup = automatic
Apr 12 03:35:05 user nova-compute[72081]: node.leading_login = No
Apr 12 03:35:05 user nova-compute[72081]: iface.iscsi_ifacename = default
Apr 12 03:35:05 user nova-compute[72081]: iface.net_ifacename = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.ipaddress = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.prefix_len = 0
Apr 12 03:35:05 user nova-compute[72081]: iface.hwaddress = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.transport_name = tcp
Apr 12 03:35:05 user nova-compute[72081]: iface.initiatorname = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.state = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.vlan_id = 0
Apr 12 03:35:05 user nova-compute[72081]: iface.vlan_priority = 0
Apr 12 03:35:05 user nova-compute[72081]: iface.vlan_state = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.iface_num = 0
Apr 12 03:35:05 user nova-compute[72081]: iface.mtu = 0
Apr 12 03:35:05 user nova-compute[72081]: iface.port = 0
Apr 12 03:35:05 user nova-compute[72081]: iface.bootproto = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.subnet_mask = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.gateway = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.dhcp_alt_client_id_state = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.dhcp_alt_client_id = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.dhcp_dns = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.dhcp_learn_iqn = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.dhcp_req_vendor_id_state = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.dhcp_vendor_id_state = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.dhcp_vendor_id = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.dhcp_slp_da = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.fragmentation = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.gratuitous_arp = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.incoming_forwarding = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.tos_state = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.tos = 0
Apr 12 03:35:05 user nova-compute[72081]: iface.ttl = 0
Apr 12 03:35:05 user nova-compute[72081]: iface.delayed_ack = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.tcp_nagle = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.tcp_wsf_state = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.tcp_wsf = 0
Apr 12 03:35:05 user nova-compute[72081]: iface.tcp_timer_scale = 0
Apr 12 03:35:05 user nova-compute[72081]: iface.tcp_timestamp = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.redirect = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.def_task_mgmt_timeout = 0
Apr 12 03:35:05 user nova-compute[72081]: iface.header_digest = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.data_digest = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.immediate_data = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.initial_r2t = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.data_seq_inorder = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.data_pdu_inorder = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.erl = 0
Apr 12 03:35:05 user nova-compute[72081]: iface.max_receive_data_len = 0
Apr 12 03:35:05 user nova-compute[72081]: iface.first_burst_len = 0
Apr 12 03:35:05 user nova-compute[72081]: iface.max_outstanding_r2t = 0
Apr 12 03:35:05 user nova-compute[72081]: iface.max_burst_len = 0
Apr 12 03:35:05 user nova-compute[72081]: iface.chap_auth = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.bidi_chap = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.strict_login_compliance = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.discovery_auth = <empty>
Apr 12 03:35:05 user nova-compute[72081]: iface.discovery_logout = <empty>
Apr 12 03:35:05 user nova-compute[72081]: node.discovery_address = <empty>
Apr 12 03:35:05 user nova-compute[72081]: node.discovery_port = 0
Apr 12 03:35:05 user nova-compute[72081]: node.discovery_type = static
Apr 12 03:35:05 user nova-compute[72081]: node.session.initial_cmdsn = 0
Apr 12 03:35:05 user nova-compute[72081]: node.session.initial_login_retry_max = 8
Apr 12 03:35:05 user nova-compute[72081]: node.session.xmit_thread_priority = -20
Apr 12 03:35:05 user nova-compute[72081]: node.session.cmds_max = 128
Apr 12 03:35:05 user nova-compute[72081]: node.session.queue_depth = 32
Apr 12 03:35:05 user nova-compute[72081]: node.session.nr_sessions = 1
Apr 12 03:35:05 user nova-compute[72081]: node.session.auth.authmethod = CHAP
Apr 12 03:35:05 user nova-compute[72081]: node.session.auth.username = FfmHBZWX
Apr 12 03:35:05 user nova-compute[72081]: node.session.auth.password = ***
Apr 12 03:35:05 user nova-compute[72081]: node.session.auth.username_in = <empty>
Apr 12 03:35:05 user nova-compute[72081]: node.session.auth.password_in = <empty>
Apr 12 03:35:05 user nova-compute[72081]: node.session.auth.chap_algs = MD5
Apr 12 03:35:05 user nova-compute[72081]: node.session.timeo.replacement_timeout = 120
Apr 12 03:35:05 user nova-compute[72081]: node.session.err_timeo.abort_timeout = 15
Apr 12 03:35:05 user nova-compute[72081]: node.session.err_timeo.lu_reset_timeout = 30
Apr 12 03:35:05 user nova-compute[72081]: node.session.err_timeo.tgt_reset_timeout = 30
Apr 12 03:35:05 user nova-compute[72081]: node.session.err_timeo.host_reset_timeout = 60
Apr 12 03:35:05 user nova-compute[72081]: node.session.iscsi.FastAbort = Yes
Apr 12 03:35:05 user nova-compute[72081]: node.session.iscsi.InitialR2T = No
Apr 12 03:35:05 user nova-compute[72081]: node.session.iscsi.ImmediateData = Yes
Apr 12 03:35:05 user nova-compute[72081]: node.session.iscsi.FirstBurstLength = 262144
Apr 12 03:35:05 user nova-compute[72081]: node.session.iscsi.MaxBurstLength = 16776192
Apr 12 03:35:05 user nova-compute[72081]: node.session.iscsi.DefaultTime2Retain = 0
Apr 12 03:35:05 user nova-compute[72081]: node.session.iscsi.DefaultTime2Wait = 2
Apr 12 03:35:05 user nova-compute[72081]: node.session.iscsi.MaxConnections = 1
Apr 12 03:35:05 user nova-compute[72081]: node.session.iscsi.MaxOutstandingR2T = 1
Apr 12 03:35:05 user nova-compute[72081]: node.session.iscsi.ERL = 0
Apr 12 03:35:05 user nova-compute[72081]: node.session.scan = manual
Apr 12 03:35:05 user nova-compute[72081]: node.session.reopen_max = 0
Apr 12 03:35:05 user nova-compute[72081]: node.conn[0].address = 172.16.0.220
Apr 12 03:35:05 user nova-compute[72081]: node.conn[0].port = 3260
Apr 12 03:35:05 user nova-compute[72081]: node.conn[0].startup = manual
Apr 12 03:35:05 user nova-compute[72081]: node.conn[0].tcp.window_size = 524288
Apr 12 03:35:05 user nova-compute[72081]: node.conn[0].tcp.type_of_service = 0
Apr 12 03:35:05 user nova-compute[72081]: node.conn[0].timeo.logout_timeout = 15
Apr 12 03:35:05 user nova-compute[72081]: node.conn[0].timeo.login_timeout = 15
Apr 12 03:35:05 user nova-compute[72081]: node.conn[0].timeo.auth_timeout = 45
Apr 12 03:35:05 user nova-compute[72081]: node.conn[0].timeo.noop_out_interval = 5
Apr 12 03:35:05 user nova-compute[72081]: node.conn[0].timeo.noop_out_timeout = 5
Apr 12 03:35:05 user nova-compute[72081]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0
Apr 12 03:35:05 user nova-compute[72081]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144
Apr 12 03:35:05 user nova-compute[72081]: node.conn[0].iscsi.HeaderDigest = None
Apr 12 03:35:05 user nova-compute[72081]: node.conn[0].iscsi.DataDigest = None
Apr 12 03:35:05 user nova-compute[72081]: node.conn[0].iscsi.IFMarker = No
Apr 12 03:35:05 user nova-compute[72081]: node.conn[0].iscsi.OFMarker = No
Apr 12 03:35:05 user nova-compute[72081]: # END RECORD
Apr 12 03:35:05 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[71ba8e65-85e4-4fbc-9152-6b764bc7d8de]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1cd1c4d9-3d40-40f6-b73f-b5b84b682164]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v FfmHBZWX {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v FfmHBZWX" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3ff80b0a-8200-43f5-92bf-006514afc233]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'FfmHBZWX'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[59fb8fc7-a6fb-4221-bc06-acf21ec6d1fb]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json" returned: 0 in 0.165s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[498e6749-3f93-4116-aae3-d9b3e714fcb7]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:35:05 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:35:05 user nova-compute[72081]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f (non-flash)
Apr 12 03:35:05 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:35:05 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:35:05 user nova-compute[72081]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f (non-flash)
Apr 12 03:35:05 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.216s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] HCTL ('33', '0', '0', 0) found on session 14 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Scanning host 33 c: 0, t: 0, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f171411c-6a67-43b2-99f2-d8df6792a8d6]: (4, ('0 0 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Searching for a device in session 14 and hctl ('33', '0', '0', 0) yield: sdb {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FfmHBZWX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 0.246s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] <== connect_volume: return (248ms) {'type': 'block', 'scsi_wwn': '23235646631346235', 'path': '/dev/sdb'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23235646631346235', 'path': '/dev/sdb'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:33:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-206381688',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-206381688',id=13,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:33:27Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='6e1fa352a8c144a1aada7ce0d8818429',ramdisk_id='',reservation_id='r-p09cbbxf',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-590302458',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:33:27Z,user_data=None,user_id='d99be569057f432484e4a60b46882bbd',uuid=bcd267d2-5b2c-4ed2-b634-0210fae94dfd,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "address": "fa:16:3e:9d:0c:c2", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "vif_mac": "fa:16:3e:9d:0c:c2"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa4385731-05", "ovs_interfaceid": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converting VIF {"id": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "address": "fa:16:3e:9d:0c:c2", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "vif_mac": "fa:16:3e:9d:0c:c2"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa4385731-05", "ovs_interfaceid": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:9d:0c:c2,bridge_name='br-int',has_traffic_filtering=True,id=a4385731-05f2-4eac-a5bb-9a987a3f5472,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa4385731-05') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lazy-loading 'pci_devices' on Instance uuid bcd267d2-5b2c-4ed2-b634-0210fae94dfd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:35:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:35:05 user nova-compute[72081]:   <uuid>bcd267d2-5b2c-4ed2-b634-0210fae94dfd</uuid>
Apr 12 03:35:05 user nova-compute[72081]:   <name>instance-0000000d</name>
Apr 12 03:35:05 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:35:05 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:35:05 user nova-compute[72081]:   <metadata>
Apr 12 03:35:05 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:35:05 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:35:05 user nova-compute[72081]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-206381688</nova:name>
Apr 12 03:35:05 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:35:05</nova:creationTime>
Apr 12 03:35:05 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:35:05 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:35:05 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:35:05 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:35:05 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:35:05 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:35:05 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:35:05 user nova-compute[72081]:       <nova:owner>
Apr 12 03:35:05 user nova-compute[72081]:         <nova:user uuid="d99be569057f432484e4a60b46882bbd">tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member</nova:user>
Apr 12 03:35:05 user nova-compute[72081]:         <nova:project uuid="6e1fa352a8c144a1aada7ce0d8818429">tempest-ServerBootFromVolumeStableRescueTest-590302458</nova:project>
Apr 12 03:35:05 user nova-compute[72081]:       </nova:owner>
Apr 12 03:35:05 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:35:05 user nova-compute[72081]:       <nova:ports>
Apr 12 03:35:05 user nova-compute[72081]:         <nova:port uuid="a4385731-05f2-4eac-a5bb-9a987a3f5472">
Apr 12 03:35:05 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Apr 12 03:35:05 user nova-compute[72081]:         </nova:port>
Apr 12 03:35:05 user nova-compute[72081]:       </nova:ports>
Apr 12 03:35:05 user nova-compute[72081]:     </nova:instance>
Apr 12 03:35:05 user nova-compute[72081]:   </metadata>
Apr 12 03:35:05 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:35:05 user nova-compute[72081]:     <system>
Apr 12 03:35:05 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:35:05 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:35:05 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:35:05 user nova-compute[72081]:       <entry name="serial">bcd267d2-5b2c-4ed2-b634-0210fae94dfd</entry>
Apr 12 03:35:05 user nova-compute[72081]:       <entry name="uuid">bcd267d2-5b2c-4ed2-b634-0210fae94dfd</entry>
Apr 12 03:35:05 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:35:05 user nova-compute[72081]:     </system>
Apr 12 03:35:05 user nova-compute[72081]:   </sysinfo>
Apr 12 03:35:05 user nova-compute[72081]:   <os>
Apr 12 03:35:05 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:35:05 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:35:05 user nova-compute[72081]:   </os>
Apr 12 03:35:05 user nova-compute[72081]:   <features>
Apr 12 03:35:05 user nova-compute[72081]:     <acpi/>
Apr 12 03:35:05 user nova-compute[72081]:     <apic/>
Apr 12 03:35:05 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:35:05 user nova-compute[72081]:   </features>
Apr 12 03:35:05 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:35:05 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:35:05 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:35:05 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:35:05 user nova-compute[72081]:   </clock>
Apr 12 03:35:05 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:35:05 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:35:05 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:35:05 user nova-compute[72081]:   </cpu>
Apr 12 03:35:05 user nova-compute[72081]:   <devices>
Apr 12 03:35:05 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:35:05 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:35:05 user nova-compute[72081]:       <source dev="/dev/sdb"/>
Apr 12 03:35:05 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:35:05 user nova-compute[72081]:       <serial>c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2</serial>
Apr 12 03:35:05 user nova-compute[72081]:     </disk>
Apr 12 03:35:05 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:35:05 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:35:05 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/bcd267d2-5b2c-4ed2-b634-0210fae94dfd/disk.rescue"/>
Apr 12 03:35:05 user nova-compute[72081]:       <target dev="vdb" bus="virtio"/>
Apr 12 03:35:05 user nova-compute[72081]:       <boot order="1"/>
Apr 12 03:35:05 user nova-compute[72081]:     </disk>
Apr 12 03:35:05 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:35:05 user nova-compute[72081]:       <mac address="fa:16:3e:9d:0c:c2"/>
Apr 12 03:35:05 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:35:05 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:35:05 user nova-compute[72081]:       <target dev="tapa4385731-05"/>
Apr 12 03:35:05 user nova-compute[72081]:     </interface>
Apr 12 03:35:05 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:35:05 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/bcd267d2-5b2c-4ed2-b634-0210fae94dfd/console.log" append="off"/>
Apr 12 03:35:05 user nova-compute[72081]:     </serial>
Apr 12 03:35:05 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:35:05 user nova-compute[72081]:     <video>
Apr 12 03:35:05 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:35:05 user nova-compute[72081]:     </video>
Apr 12 03:35:05 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:35:05 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:35:05 user nova-compute[72081]:     </rng>
Apr 12 03:35:05 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:35:05 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:35:05 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:35:05 user nova-compute[72081]:     </memballoon>
Apr 12 03:35:05 user nova-compute[72081]:   </devices>
Apr 12 03:35:05 user nova-compute[72081]: </domain>
Apr 12 03:35:05 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:35:05 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Instance destroyed successfully.
Apr 12 03:35:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] No VIF found with MAC fa:16:3e:9d:0c:c2, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:35:06 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:35:06 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:35:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=7737MB free_disk=26.14974594116211GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "49d33636-1dbb-4863-b0f3-9f1578e23e67" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "49d33636-1dbb-4863-b0f3-9f1578e23e67" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG nova.compute.manager [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 56834002-7593-4592-a86d-23bb90d00be0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 2d0904c3-74e0-47bb-a832-758a7035b2f5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 72ba1218-065d-40c4-8503-7c4155e47503 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance bcd267d2-5b2c-4ed2-b634-0210fae94dfd actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 48977eb2-ce43-4577-a1cb-340e35703dbd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance b23d4bcd-989a-45e1-89a3-6e3d30229a32 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 23367b24-0a3b-454d-901d-8a3e5396333a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 49d33636-1dbb-4863-b0f3-9f1578e23e67 has been scheduled to this compute host, the scheduler has made an allocation against this compute node but the instance has yet to start. Skipping heal of allocation: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1689}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=1664MB phys_disk=40GB used_disk=8GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.519s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.349s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:35:06 user nova-compute[72081]: INFO nova.compute.claims [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Claim successful on node user
Apr 12 03:35:06 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.442s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:35:06 user nova-compute[72081]: DEBUG nova.compute.manager [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:35:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:35:07 user nova-compute[72081]: DEBUG nova.network.neutron [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:35:07 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:35:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:35:07 user nova-compute[72081]: INFO nova.virt.block_device [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Booting with volume snapshot d390d8d2-1444-42f4-8136-2216da391ac2 at /dev/vda
Apr 12 03:35:07 user nova-compute[72081]: DEBUG nova.policy [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'b00b348933224558aad1cb12921f100f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '003688353e2f4b36831c574ebaf65173', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:35:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:07 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:35:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:35:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Rebuilding the list of instances to heal {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 12 03:35:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Skipping network cache update for instance because it is Building. {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9805}}
Apr 12 03:35:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-56834002-7593-4592-a86d-23bb90d00be0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:35:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-56834002-7593-4592-a86d-23bb90d00be0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:35:07 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 03:35:07 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lazy-loading 'info_cache' on Instance uuid 56834002-7593-4592-a86d-23bb90d00be0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:35:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-d01dee42-b155-4e4f-bfdf-8595db3f3d52 req-f30f40b0-45a8-41c7-95b7-99a005650a67 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:35:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d01dee42-b155-4e4f-bfdf-8595db3f3d52 req-f30f40b0-45a8-41c7-95b7-99a005650a67 service nova] Acquiring lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:35:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d01dee42-b155-4e4f-bfdf-8595db3f3d52 req-f30f40b0-45a8-41c7-95b7-99a005650a67 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:35:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d01dee42-b155-4e4f-bfdf-8595db3f3d52 req-f30f40b0-45a8-41c7-95b7-99a005650a67 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:35:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-d01dee42-b155-4e4f-bfdf-8595db3f3d52 req-f30f40b0-45a8-41c7-95b7-99a005650a67 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] No waiting events found dispatching network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:35:07 user nova-compute[72081]: WARNING nova.compute.manager [req-d01dee42-b155-4e4f-bfdf-8595db3f3d52 req-f30f40b0-45a8-41c7-95b7-99a005650a67 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received unexpected event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 for instance with vm_state active and task_state rescuing.
Apr 12 03:35:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:09 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Updating instance_info_cache with network_info: [{"id": "8f7b221a-25ae-41f9-847d-5aedf2d93a71", "address": "fa:16:3e:e7:22:1b", "network": {"id": "1a483654-b740-4544-84c9-53a41c11fa88", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1141588183-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "83a7ef3f97fd47849973d4e4482fb5f9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f7b221a-25", "ovs_interfaceid": "8f7b221a-25ae-41f9-847d-5aedf2d93a71", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:35:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-56834002-7593-4592-a86d-23bb90d00be0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:35:09 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 03:35:09 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:35:09 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:35:09 user nova-compute[72081]: DEBUG nova.network.neutron [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Successfully created port: 93c27018-4ee9-42be-81a1-2b9377a689f5 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:35:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-b80962a6-598d-4d01-b1ca-d945a7cdfd33 req-1e496ed6-7bbb-4c87-9ac5-69276f4525c9 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:35:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b80962a6-598d-4d01-b1ca-d945a7cdfd33 req-1e496ed6-7bbb-4c87-9ac5-69276f4525c9 service nova] Acquiring lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:35:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b80962a6-598d-4d01-b1ca-d945a7cdfd33 req-1e496ed6-7bbb-4c87-9ac5-69276f4525c9 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:35:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b80962a6-598d-4d01-b1ca-d945a7cdfd33 req-1e496ed6-7bbb-4c87-9ac5-69276f4525c9 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:35:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-b80962a6-598d-4d01-b1ca-d945a7cdfd33 req-1e496ed6-7bbb-4c87-9ac5-69276f4525c9 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] No waiting events found dispatching network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:35:09 user nova-compute[72081]: WARNING nova.compute.manager [req-b80962a6-598d-4d01-b1ca-d945a7cdfd33 req-1e496ed6-7bbb-4c87-9ac5-69276f4525c9 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received unexpected event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 for instance with vm_state active and task_state rescuing.
Apr 12 03:35:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Removed pending event for bcd267d2-5b2c-4ed2-b634-0210fae94dfd due to event {{(pid=72081) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 12 03:35:10 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270510.1231434, bcd267d2-5b2c-4ed2-b634-0210fae94dfd => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:35:10 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] VM Resumed (Lifecycle Event)
Apr 12 03:35:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9d691e4a-b2ae-4c44-8a3f-aa85fe7baa87 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:35:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:35:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:35:10 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] During sync_power_state the instance has a pending task (rescuing). Skip.
Apr 12 03:35:10 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270510.1247683, bcd267d2-5b2c-4ed2-b634-0210fae94dfd => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:35:10 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] VM Started (Lifecycle Event)
Apr 12 03:35:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:35:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Synchronizing instance power state after lifecycle event "Started"; current vm_state: rescued, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:35:10 user nova-compute[72081]: DEBUG nova.network.neutron [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Successfully updated port: 93c27018-4ee9-42be-81a1-2b9377a689f5 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:35:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "refresh_cache-49d33636-1dbb-4863-b0f3-9f1578e23e67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:35:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquired lock "refresh_cache-49d33636-1dbb-4863-b0f3-9f1578e23e67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:35:10 user nova-compute[72081]: DEBUG nova.network.neutron [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:35:10 user nova-compute[72081]: DEBUG nova.network.neutron [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:35:11 user nova-compute[72081]: DEBUG nova.compute.manager [req-63b8d5ce-cc9d-45ec-92bd-b93725d76860 req-7c842357-0eb5-479c-a5e9-d0c6b6f94d5e service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received event network-changed-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:35:11 user nova-compute[72081]: DEBUG nova.compute.manager [req-63b8d5ce-cc9d-45ec-92bd-b93725d76860 req-7c842357-0eb5-479c-a5e9-d0c6b6f94d5e service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Refreshing instance network info cache due to event network-changed-18f9f68c-eadd-4582-8f16-b88a13496852. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:35:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-63b8d5ce-cc9d-45ec-92bd-b93725d76860 req-7c842357-0eb5-479c-a5e9-d0c6b6f94d5e service nova] Acquiring lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:35:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-63b8d5ce-cc9d-45ec-92bd-b93725d76860 req-7c842357-0eb5-479c-a5e9-d0c6b6f94d5e service nova] Acquired lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:35:11 user nova-compute[72081]: DEBUG nova.network.neutron [req-63b8d5ce-cc9d-45ec-92bd-b93725d76860 req-7c842357-0eb5-479c-a5e9-d0c6b6f94d5e service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Refreshing network info cache for port 18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:35:11 user nova-compute[72081]: DEBUG nova.network.neutron [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Updating instance_info_cache with network_info: [{"id": "93c27018-4ee9-42be-81a1-2b9377a689f5", "address": "fa:16:3e:85:8d:3c", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap93c27018-4e", "ovs_interfaceid": "93c27018-4ee9-42be-81a1-2b9377a689f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:35:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Releasing lock "refresh_cache-49d33636-1dbb-4863-b0f3-9f1578e23e67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:35:11 user nova-compute[72081]: DEBUG nova.compute.manager [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Instance network_info: |[{"id": "93c27018-4ee9-42be-81a1-2b9377a689f5", "address": "fa:16:3e:85:8d:3c", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap93c27018-4e", "ovs_interfaceid": "93c27018-4ee9-42be-81a1-2b9377a689f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:35:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:11 user nova-compute[72081]: DEBUG nova.compute.manager [req-2016f0ba-eb45-40ac-8892-440b5689cf10 req-12153491-ef46-401a-aa8c-fde4aa88b7d2 service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Received event network-changed-93c27018-4ee9-42be-81a1-2b9377a689f5 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:35:11 user nova-compute[72081]: DEBUG nova.compute.manager [req-2016f0ba-eb45-40ac-8892-440b5689cf10 req-12153491-ef46-401a-aa8c-fde4aa88b7d2 service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Refreshing instance network info cache due to event network-changed-93c27018-4ee9-42be-81a1-2b9377a689f5. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:35:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2016f0ba-eb45-40ac-8892-440b5689cf10 req-12153491-ef46-401a-aa8c-fde4aa88b7d2 service nova] Acquiring lock "refresh_cache-49d33636-1dbb-4863-b0f3-9f1578e23e67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:35:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2016f0ba-eb45-40ac-8892-440b5689cf10 req-12153491-ef46-401a-aa8c-fde4aa88b7d2 service nova] Acquired lock "refresh_cache-49d33636-1dbb-4863-b0f3-9f1578e23e67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:35:11 user nova-compute[72081]: DEBUG nova.network.neutron [req-2016f0ba-eb45-40ac-8892-440b5689cf10 req-12153491-ef46-401a-aa8c-fde4aa88b7d2 service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Refreshing network info cache for port 93c27018-4ee9-42be-81a1-2b9377a689f5 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:35:12 user nova-compute[72081]: DEBUG nova.network.neutron [req-63b8d5ce-cc9d-45ec-92bd-b93725d76860 req-7c842357-0eb5-479c-a5e9-d0c6b6f94d5e service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Updated VIF entry in instance network info cache for port 18f9f68c-eadd-4582-8f16-b88a13496852. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:35:12 user nova-compute[72081]: DEBUG nova.network.neutron [req-63b8d5ce-cc9d-45ec-92bd-b93725d76860 req-7c842357-0eb5-479c-a5e9-d0c6b6f94d5e service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Updating instance_info_cache with network_info: [{"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:35:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-63b8d5ce-cc9d-45ec-92bd-b93725d76860 req-7c842357-0eb5-479c-a5e9-d0c6b6f94d5e service nova] Releasing lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:35:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:13 user nova-compute[72081]: DEBUG os_brick.utils [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:35:13 user nova-compute[72081]: DEBUG nova.network.neutron [req-2016f0ba-eb45-40ac-8892-440b5689cf10 req-12153491-ef46-401a-aa8c-fde4aa88b7d2 service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Updated VIF entry in instance network info cache for port 93c27018-4ee9-42be-81a1-2b9377a689f5. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:35:13 user nova-compute[72081]: DEBUG nova.network.neutron [req-2016f0ba-eb45-40ac-8892-440b5689cf10 req-12153491-ef46-401a-aa8c-fde4aa88b7d2 service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Updating instance_info_cache with network_info: [{"id": "93c27018-4ee9-42be-81a1-2b9377a689f5", "address": "fa:16:3e:85:8d:3c", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap93c27018-4e", "ovs_interfaceid": "93c27018-4ee9-42be-81a1-2b9377a689f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:35:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2016f0ba-eb45-40ac-8892-440b5689cf10 req-12153491-ef46-401a-aa8c-fde4aa88b7d2 service nova] Releasing lock "refresh_cache-49d33636-1dbb-4863-b0f3-9f1578e23e67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:35:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:13 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[11907f88-87c3-487b-989c-407fdd71f76e]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:13 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[280fec09-c67c-4c39-82cf-3b3ac4132868]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:13 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[642743f5-6d1e-4bcf-9234-82eeb820f226]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:13 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ef893818-71bd-4e35-a937-221088ffd8f6]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:35:13 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:35:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:35:13 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:35:13 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b5a9a199-c0fe-43f0-96c4-4a2c6b893e33]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:13 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:35:13 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:35:13 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:35:13 user nova-compute[72081]: DEBUG os_brick.utils [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] <== get_connector_properties: return (143ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:35:13 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Updating existing volume attachment record: 7b83dead-4a54-4c10-ba94-6d78ec8bce9a {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:35:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquiring lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:35:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:35:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:35:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:35:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:35:14 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:35:14 user nova-compute[72081]: INFO nova.compute.claims [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Claim successful on node user
Apr 12 03:35:15 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:35:15 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:35:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.415s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:35:15 user nova-compute[72081]: DEBUG nova.compute.manager [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:35:15 user nova-compute[72081]: DEBUG nova.compute.manager [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:35:15 user nova-compute[72081]: DEBUG nova.network.neutron [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:35:15 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:35:15 user nova-compute[72081]: DEBUG nova.compute.manager [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:35:15 user nova-compute[72081]: DEBUG nova.policy [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'e2f496e5fe694d46a9803f797743a7dc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4cce206e2b94b03ade2d310373c5ca2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:35:15 user nova-compute[72081]: DEBUG nova.compute.manager [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:35:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:35:15 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Creating image(s)
Apr 12 03:35:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquiring lock "/opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:35:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "/opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:35:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "/opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:35:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.178s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:35:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:35:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.243s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk 1073741824" returned: 0 in 0.053s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.304s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:35:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:16 user nova-compute[72081]: DEBUG nova.network.neutron [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Successfully created port: 892ff129-c9ca-4295-be61-993b81b855fb {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:35:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.141s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:16 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Checking if we can resize image /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:35:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:16 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Cannot resize image /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:35:16 user nova-compute[72081]: DEBUG nova.objects.instance [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lazy-loading 'migration_context' on Instance uuid a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:35:16 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:35:16 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Ensure instance console log exists: /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:35:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:35:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:35:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:35:16 user nova-compute[72081]: DEBUG nova.network.neutron [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Successfully updated port: 892ff129-c9ca-4295-be61-993b81b855fb {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.compute.manager [req-539a16d2-79c8-452d-a7b0-ffc73878f740 req-994d363c-a63e-462b-8b54-a3e2cc64ea85 service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Received event network-changed-892ff129-c9ca-4295-be61-993b81b855fb {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.compute.manager [req-539a16d2-79c8-452d-a7b0-ffc73878f740 req-994d363c-a63e-462b-8b54-a3e2cc64ea85 service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Refreshing instance network info cache due to event network-changed-892ff129-c9ca-4295-be61-993b81b855fb. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-539a16d2-79c8-452d-a7b0-ffc73878f740 req-994d363c-a63e-462b-8b54-a3e2cc64ea85 service nova] Acquiring lock "refresh_cache-a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-539a16d2-79c8-452d-a7b0-ffc73878f740 req-994d363c-a63e-462b-8b54-a3e2cc64ea85 service nova] Acquired lock "refresh_cache-a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.network.neutron [req-539a16d2-79c8-452d-a7b0-ffc73878f740 req-994d363c-a63e-462b-8b54-a3e2cc64ea85 service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Refreshing network info cache for port 892ff129-c9ca-4295-be61-993b81b855fb {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.network.neutron [req-539a16d2-79c8-452d-a7b0-ffc73878f740 req-994d363c-a63e-462b-8b54-a3e2cc64ea85 service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquiring lock "refresh_cache-a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.network.neutron [req-539a16d2-79c8-452d-a7b0-ffc73878f740 req-994d363c-a63e-462b-8b54-a3e2cc64ea85 service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-539a16d2-79c8-452d-a7b0-ffc73878f740 req-994d363c-a63e-462b-8b54-a3e2cc64ea85 service nova] Releasing lock "refresh_cache-a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquired lock "refresh_cache-a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.network.neutron [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.network.neutron [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.network.neutron [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Updating instance_info_cache with network_info: [{"id": "892ff129-c9ca-4295-be61-993b81b855fb", "address": "fa:16:3e:f0:b1:a0", "network": {"id": "263f9508-8639-4289-9210-f9807acf6766", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-337713645-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f4cce206e2b94b03ade2d310373c5ca2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap892ff129-c9", "ovs_interfaceid": "892ff129-c9ca-4295-be61-993b81b855fb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Releasing lock "refresh_cache-a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.compute.manager [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Instance network_info: |[{"id": "892ff129-c9ca-4295-be61-993b81b855fb", "address": "fa:16:3e:f0:b1:a0", "network": {"id": "263f9508-8639-4289-9210-f9807acf6766", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-337713645-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f4cce206e2b94b03ade2d310373c5ca2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap892ff129-c9", "ovs_interfaceid": "892ff129-c9ca-4295-be61-993b81b855fb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Start _get_guest_xml network_info=[{"id": "892ff129-c9ca-4295-be61-993b81b855fb", "address": "fa:16:3e:f0:b1:a0", "network": {"id": "263f9508-8639-4289-9210-f9807acf6766", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-337713645-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f4cce206e2b94b03ade2d310373c5ca2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap892ff129-c9", "ovs_interfaceid": "892ff129-c9ca-4295-be61-993b81b855fb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:35:17 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:35:17 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:35:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-2065419898',display_name='tempest-VolumesAdminNegativeTest-server-2065419898',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-2065419898',id=19,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEZmg9av5nHzz+WM/CeO9E4jVqkDumG/6+9A6cHxDvsmxb0GgBIDFr9i3AR8FMpBKXAtQyO3hr0qIi3g2HX/TjWxD4YibE1LO5tXDCbvlMnQhnENYSPZrfx3AZVOxrgL2g==',key_name='tempest-keypair-315148094',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f4cce206e2b94b03ade2d310373c5ca2',ramdisk_id='',reservation_id='r-i8pcads2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesAdminNegativeTest-2105871059',owner_user_name='tempest-VolumesAdminNegativeTest-2105871059-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:35:15Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e2f496e5fe694d46a9803f797743a7dc',uuid=a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "892ff129-c9ca-4295-be61-993b81b855fb", "address": "fa:16:3e:f0:b1:a0", "network": {"id": "263f9508-8639-4289-9210-f9807acf6766", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-337713645-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f4cce206e2b94b03ade2d310373c5ca2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap892ff129-c9", "ovs_interfaceid": "892ff129-c9ca-4295-be61-993b81b855fb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Converting VIF {"id": "892ff129-c9ca-4295-be61-993b81b855fb", "address": "fa:16:3e:f0:b1:a0", "network": {"id": "263f9508-8639-4289-9210-f9807acf6766", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-337713645-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f4cce206e2b94b03ade2d310373c5ca2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap892ff129-c9", "ovs_interfaceid": "892ff129-c9ca-4295-be61-993b81b855fb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f0:b1:a0,bridge_name='br-int',has_traffic_filtering=True,id=892ff129-c9ca-4295-be61-993b81b855fb,network=Network(263f9508-8639-4289-9210-f9807acf6766),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap892ff129-c9') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.objects.instance [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lazy-loading 'pci_devices' on Instance uuid a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:35:17 user nova-compute[72081]:   <uuid>a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6</uuid>
Apr 12 03:35:17 user nova-compute[72081]:   <name>instance-00000013</name>
Apr 12 03:35:17 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:35:17 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:35:17 user nova-compute[72081]:   <metadata>
Apr 12 03:35:17 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:35:17 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:35:17 user nova-compute[72081]:       <nova:name>tempest-VolumesAdminNegativeTest-server-2065419898</nova:name>
Apr 12 03:35:17 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:35:17</nova:creationTime>
Apr 12 03:35:17 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:35:17 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:35:17 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:35:17 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:35:17 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:35:17 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:35:17 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:35:17 user nova-compute[72081]:       <nova:owner>
Apr 12 03:35:17 user nova-compute[72081]:         <nova:user uuid="e2f496e5fe694d46a9803f797743a7dc">tempest-VolumesAdminNegativeTest-2105871059-project-member</nova:user>
Apr 12 03:35:17 user nova-compute[72081]:         <nova:project uuid="f4cce206e2b94b03ade2d310373c5ca2">tempest-VolumesAdminNegativeTest-2105871059</nova:project>
Apr 12 03:35:17 user nova-compute[72081]:       </nova:owner>
Apr 12 03:35:17 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:35:17 user nova-compute[72081]:       <nova:ports>
Apr 12 03:35:17 user nova-compute[72081]:         <nova:port uuid="892ff129-c9ca-4295-be61-993b81b855fb">
Apr 12 03:35:17 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Apr 12 03:35:17 user nova-compute[72081]:         </nova:port>
Apr 12 03:35:17 user nova-compute[72081]:       </nova:ports>
Apr 12 03:35:17 user nova-compute[72081]:     </nova:instance>
Apr 12 03:35:17 user nova-compute[72081]:   </metadata>
Apr 12 03:35:17 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:35:17 user nova-compute[72081]:     <system>
Apr 12 03:35:17 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:35:17 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:35:17 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:35:17 user nova-compute[72081]:       <entry name="serial">a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6</entry>
Apr 12 03:35:17 user nova-compute[72081]:       <entry name="uuid">a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6</entry>
Apr 12 03:35:17 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:35:17 user nova-compute[72081]:     </system>
Apr 12 03:35:17 user nova-compute[72081]:   </sysinfo>
Apr 12 03:35:17 user nova-compute[72081]:   <os>
Apr 12 03:35:17 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:35:17 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:35:17 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:35:17 user nova-compute[72081]:   </os>
Apr 12 03:35:17 user nova-compute[72081]:   <features>
Apr 12 03:35:17 user nova-compute[72081]:     <acpi/>
Apr 12 03:35:17 user nova-compute[72081]:     <apic/>
Apr 12 03:35:17 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:35:17 user nova-compute[72081]:   </features>
Apr 12 03:35:17 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:35:17 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:35:17 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:35:17 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:35:17 user nova-compute[72081]:   </clock>
Apr 12 03:35:17 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:35:17 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:35:17 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:35:17 user nova-compute[72081]:   </cpu>
Apr 12 03:35:17 user nova-compute[72081]:   <devices>
Apr 12 03:35:17 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:35:17 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:35:17 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk"/>
Apr 12 03:35:17 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:35:17 user nova-compute[72081]:     </disk>
Apr 12 03:35:17 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:35:17 user nova-compute[72081]:       <mac address="fa:16:3e:f0:b1:a0"/>
Apr 12 03:35:17 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:35:17 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:35:17 user nova-compute[72081]:       <target dev="tap892ff129-c9"/>
Apr 12 03:35:17 user nova-compute[72081]:     </interface>
Apr 12 03:35:17 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:35:17 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/console.log" append="off"/>
Apr 12 03:35:17 user nova-compute[72081]:     </serial>
Apr 12 03:35:17 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:35:17 user nova-compute[72081]:     <video>
Apr 12 03:35:17 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:35:17 user nova-compute[72081]:     </video>
Apr 12 03:35:17 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:35:17 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:35:17 user nova-compute[72081]:     </rng>
Apr 12 03:35:17 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:35:17 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:35:17 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:35:17 user nova-compute[72081]:     </memballoon>
Apr 12 03:35:17 user nova-compute[72081]:   </devices>
Apr 12 03:35:17 user nova-compute[72081]: </domain>
Apr 12 03:35:17 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:35:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-2065419898',display_name='tempest-VolumesAdminNegativeTest-server-2065419898',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-2065419898',id=19,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEZmg9av5nHzz+WM/CeO9E4jVqkDumG/6+9A6cHxDvsmxb0GgBIDFr9i3AR8FMpBKXAtQyO3hr0qIi3g2HX/TjWxD4YibE1LO5tXDCbvlMnQhnENYSPZrfx3AZVOxrgL2g==',key_name='tempest-keypair-315148094',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f4cce206e2b94b03ade2d310373c5ca2',ramdisk_id='',reservation_id='r-i8pcads2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesAdminNegativeTest-2105871059',owner_user_name='tempest-VolumesAdminNegativeTest-2105871059-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:35:15Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e2f496e5fe694d46a9803f797743a7dc',uuid=a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "892ff129-c9ca-4295-be61-993b81b855fb", "address": "fa:16:3e:f0:b1:a0", "network": {"id": "263f9508-8639-4289-9210-f9807acf6766", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-337713645-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f4cce206e2b94b03ade2d310373c5ca2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap892ff129-c9", "ovs_interfaceid": "892ff129-c9ca-4295-be61-993b81b855fb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Converting VIF {"id": "892ff129-c9ca-4295-be61-993b81b855fb", "address": "fa:16:3e:f0:b1:a0", "network": {"id": "263f9508-8639-4289-9210-f9807acf6766", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-337713645-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f4cce206e2b94b03ade2d310373c5ca2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap892ff129-c9", "ovs_interfaceid": "892ff129-c9ca-4295-be61-993b81b855fb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f0:b1:a0,bridge_name='br-int',has_traffic_filtering=True,id=892ff129-c9ca-4295-be61-993b81b855fb,network=Network(263f9508-8639-4289-9210-f9807acf6766),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap892ff129-c9') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG os_vif [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f0:b1:a0,bridge_name='br-int',has_traffic_filtering=True,id=892ff129-c9ca-4295-be61-993b81b855fb,network=Network(263f9508-8639-4289-9210-f9807acf6766),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap892ff129-c9') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap892ff129-c9, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap892ff129-c9, col_values=(('external_ids', {'iface-id': '892ff129-c9ca-4295-be61-993b81b855fb', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:f0:b1:a0', 'vm-uuid': 'a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:17 user nova-compute[72081]: INFO os_vif [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f0:b1:a0,bridge_name='br-int',has_traffic_filtering=True,id=892ff129-c9ca-4295-be61-993b81b855fb,network=Network(263f9508-8639-4289-9210-f9807acf6766),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap892ff129-c9')
Apr 12 03:35:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.compute.manager [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:35:17 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Creating image(s)
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Did not create local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Ensure instance console log exists: /opt/stack/data/nova/instances/49d33636-1dbb-4863-b0f3-9f1578e23e67/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Start _get_guest_xml network_info=[{"id": "93c27018-4ee9-42be-81a1-2b9377a689f5", "address": "fa:16:3e:85:8d:3c", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap93c27018-4e", "ovs_interfaceid": "93c27018-4ee9-42be-81a1-2b9377a689f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'QptXErmF', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '49d33636-1dbb-4863-b0f3-9f1578e23e67', 'attached_at': '', 'detached_at': '', 'volume_id': '4bd5227e-f472-471d-8e4f-7088f977745f', 'serial': '4bd5227e-f472-471d-8e4f-7088f977745f'}, 'disk_bus': 'virtio', 'delete_on_termination': True, 'boot_index': 0, 'guest_format': None, 'device_type': 'disk', 'attachment_id': '7b83dead-4a54-4c10-ba94-6d78ec8bce9a', 'volume_type': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:35:17 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:35:17 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'QptXErmF', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:35:17 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:35:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:35:17 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:35:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:35:17 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] No VIF found with MAC fa:16:3e:f0:b1:a0, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3165d240-0d5d-4256-8d0c-be2cd0aff447]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:35:17 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[89bb9dfa-2af3-49b8-b66b-28df32984f58]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f] added
Apr 12 03:35:17 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7713cfe9-43a2-4bb2-92f2-9779823ec600]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7a4b9426-a5ef-4e27-ba01-b4f28a8cbe05]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f -p 172.16.0.220:3260 --op update -n node.session.auth.username -v QptXErmF {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f -p 172.16.0.220:3260 --op update -n node.session.auth.username -v QptXErmF" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c99161e9-211f-4aa0-8d8c-91666f838981]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'QptXErmF'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e1a61c99-611b-48b3-9333-65126bd5769c]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[dbd86908-4a8e-4624-9a88-6f5f2d4ca4fd]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:35:17 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:35:17 user nova-compute[72081]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f (non-flash)
Apr 12 03:35:17 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:35:17 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:35:17 user nova-compute[72081]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f (non-flash)
Apr 12 03:35:17 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:35:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f -p 172.16.0.220:3260 --login" returned: 0 in 0.066s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[67898684-529a-4e6c-afd2-3be554501ef9]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f, portal: 172.16.0.220,3260]
Apr 12 03:35:18 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f, portal: 172.16.0.220,3260] successful.
Apr 12 03:35:18 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:35:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[00adb539-ac60-4122-a1e7-518d100ddc71]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:35:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[dca1c894-d2f6-436c-943d-0087ae12e9ec]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:35:18 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:35:18 user nova-compute[72081]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f (non-flash)
Apr 12 03:35:18 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:35:18 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:35:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:35:18 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:35:18 user nova-compute[72081]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f (non-flash)
Apr 12 03:35:18 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:35:18 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:35:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.280s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:35:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:35:18 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] HCTL ('36', '-', '-', 0) found on session 17 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:35:18 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:35:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[81bee57d-57de-4763-9e87-d3f86623417a]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:18 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Searching for a device in session 17 and hctl ['36', '*', '*', 0] yield: sde {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:35:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'QptXErmF', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.323s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] <== connect_volume: return (1324ms) {'type': 'block', 'scsi_wwn': '26537316332646130', 'path': '/dev/sde'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26537316332646130', 'path': '/dev/sde'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:35:05Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1468791143',display_name='tempest-TestVolumeBootPattern-server-1468791143',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1468791143',id=18,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='003688353e2f4b36831c574ebaf65173',ramdisk_id='',reservation_id='r-v7szkyzi',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-235043461',owner_user_name='tempest-TestVolumeBootPattern-235043461-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:35:07Z,user_data=None,user_id='b00b348933224558aad1cb12921f100f',uuid=49d33636-1dbb-4863-b0f3-9f1578e23e67,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "93c27018-4ee9-42be-81a1-2b9377a689f5", "address": "fa:16:3e:85:8d:3c", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap93c27018-4e", "ovs_interfaceid": "93c27018-4ee9-42be-81a1-2b9377a689f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converting VIF {"id": "93c27018-4ee9-42be-81a1-2b9377a689f5", "address": "fa:16:3e:85:8d:3c", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap93c27018-4e", "ovs_interfaceid": "93c27018-4ee9-42be-81a1-2b9377a689f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:85:8d:3c,bridge_name='br-int',has_traffic_filtering=True,id=93c27018-4ee9-42be-81a1-2b9377a689f5,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap93c27018-4e') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG nova.objects.instance [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lazy-loading 'pci_devices' on Instance uuid 49d33636-1dbb-4863-b0f3-9f1578e23e67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:35:19 user nova-compute[72081]:   <uuid>49d33636-1dbb-4863-b0f3-9f1578e23e67</uuid>
Apr 12 03:35:19 user nova-compute[72081]:   <name>instance-00000012</name>
Apr 12 03:35:19 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:35:19 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:35:19 user nova-compute[72081]:   <metadata>
Apr 12 03:35:19 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:35:19 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:35:19 user nova-compute[72081]:       <nova:name>tempest-TestVolumeBootPattern-server-1468791143</nova:name>
Apr 12 03:35:19 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:35:17</nova:creationTime>
Apr 12 03:35:19 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:35:19 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:35:19 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:35:19 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:35:19 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:35:19 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:35:19 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:35:19 user nova-compute[72081]:       <nova:owner>
Apr 12 03:35:19 user nova-compute[72081]:         <nova:user uuid="b00b348933224558aad1cb12921f100f">tempest-TestVolumeBootPattern-235043461-project-member</nova:user>
Apr 12 03:35:19 user nova-compute[72081]:         <nova:project uuid="003688353e2f4b36831c574ebaf65173">tempest-TestVolumeBootPattern-235043461</nova:project>
Apr 12 03:35:19 user nova-compute[72081]:       </nova:owner>
Apr 12 03:35:19 user nova-compute[72081]:       <nova:ports>
Apr 12 03:35:19 user nova-compute[72081]:         <nova:port uuid="93c27018-4ee9-42be-81a1-2b9377a689f5">
Apr 12 03:35:19 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 12 03:35:19 user nova-compute[72081]:         </nova:port>
Apr 12 03:35:19 user nova-compute[72081]:       </nova:ports>
Apr 12 03:35:19 user nova-compute[72081]:     </nova:instance>
Apr 12 03:35:19 user nova-compute[72081]:   </metadata>
Apr 12 03:35:19 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:35:19 user nova-compute[72081]:     <system>
Apr 12 03:35:19 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:35:19 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:35:19 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:35:19 user nova-compute[72081]:       <entry name="serial">49d33636-1dbb-4863-b0f3-9f1578e23e67</entry>
Apr 12 03:35:19 user nova-compute[72081]:       <entry name="uuid">49d33636-1dbb-4863-b0f3-9f1578e23e67</entry>
Apr 12 03:35:19 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:35:19 user nova-compute[72081]:     </system>
Apr 12 03:35:19 user nova-compute[72081]:   </sysinfo>
Apr 12 03:35:19 user nova-compute[72081]:   <os>
Apr 12 03:35:19 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:35:19 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:35:19 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:35:19 user nova-compute[72081]:   </os>
Apr 12 03:35:19 user nova-compute[72081]:   <features>
Apr 12 03:35:19 user nova-compute[72081]:     <acpi/>
Apr 12 03:35:19 user nova-compute[72081]:     <apic/>
Apr 12 03:35:19 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:35:19 user nova-compute[72081]:   </features>
Apr 12 03:35:19 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:35:19 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:35:19 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:35:19 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:35:19 user nova-compute[72081]:   </clock>
Apr 12 03:35:19 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:35:19 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:35:19 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:35:19 user nova-compute[72081]:   </cpu>
Apr 12 03:35:19 user nova-compute[72081]:   <devices>
Apr 12 03:35:19 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:35:19 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:35:19 user nova-compute[72081]:       <source dev="/dev/sde"/>
Apr 12 03:35:19 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:35:19 user nova-compute[72081]:       <serial>4bd5227e-f472-471d-8e4f-7088f977745f</serial>
Apr 12 03:35:19 user nova-compute[72081]:     </disk>
Apr 12 03:35:19 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:35:19 user nova-compute[72081]:       <mac address="fa:16:3e:85:8d:3c"/>
Apr 12 03:35:19 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:35:19 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:35:19 user nova-compute[72081]:       <target dev="tap93c27018-4e"/>
Apr 12 03:35:19 user nova-compute[72081]:     </interface>
Apr 12 03:35:19 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:35:19 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/49d33636-1dbb-4863-b0f3-9f1578e23e67/console.log" append="off"/>
Apr 12 03:35:19 user nova-compute[72081]:     </serial>
Apr 12 03:35:19 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:35:19 user nova-compute[72081]:     <video>
Apr 12 03:35:19 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:35:19 user nova-compute[72081]:     </video>
Apr 12 03:35:19 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:35:19 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:35:19 user nova-compute[72081]:     </rng>
Apr 12 03:35:19 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:35:19 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:35:19 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:35:19 user nova-compute[72081]:     </memballoon>
Apr 12 03:35:19 user nova-compute[72081]:   </devices>
Apr 12 03:35:19 user nova-compute[72081]: </domain>
Apr 12 03:35:19 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:35:05Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1468791143',display_name='tempest-TestVolumeBootPattern-server-1468791143',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1468791143',id=18,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='003688353e2f4b36831c574ebaf65173',ramdisk_id='',reservation_id='r-v7szkyzi',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-235043461',owner_user_name='tempest-TestVolumeBootPattern-235043461-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:35:07Z,user_data=None,user_id='b00b348933224558aad1cb12921f100f',uuid=49d33636-1dbb-4863-b0f3-9f1578e23e67,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "93c27018-4ee9-42be-81a1-2b9377a689f5", "address": "fa:16:3e:85:8d:3c", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap93c27018-4e", "ovs_interfaceid": "93c27018-4ee9-42be-81a1-2b9377a689f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converting VIF {"id": "93c27018-4ee9-42be-81a1-2b9377a689f5", "address": "fa:16:3e:85:8d:3c", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap93c27018-4e", "ovs_interfaceid": "93c27018-4ee9-42be-81a1-2b9377a689f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:85:8d:3c,bridge_name='br-int',has_traffic_filtering=True,id=93c27018-4ee9-42be-81a1-2b9377a689f5,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap93c27018-4e') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG os_vif [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:85:8d:3c,bridge_name='br-int',has_traffic_filtering=True,id=93c27018-4ee9-42be-81a1-2b9377a689f5,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap93c27018-4e') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap93c27018-4e, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap93c27018-4e, col_values=(('external_ids', {'iface-id': '93c27018-4ee9-42be-81a1-2b9377a689f5', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:85:8d:3c', 'vm-uuid': '49d33636-1dbb-4863-b0f3-9f1578e23e67'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:19 user nova-compute[72081]: INFO os_vif [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:85:8d:3c,bridge_name='br-int',has_traffic_filtering=True,id=93c27018-4ee9-42be-81a1-2b9377a689f5,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap93c27018-4e')
Apr 12 03:35:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] No VIF found with MAC fa:16:3e:85:8d:3c, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG nova.compute.manager [req-ad65848b-cce6-4bc8-9a09-ac4d22892c0a req-08c60e7d-53ee-483f-9112-cdb08a396ade service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Received event network-vif-plugged-892ff129-c9ca-4295-be61-993b81b855fb {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ad65848b-cce6-4bc8-9a09-ac4d22892c0a req-08c60e7d-53ee-483f-9112-cdb08a396ade service nova] Acquiring lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ad65848b-cce6-4bc8-9a09-ac4d22892c0a req-08c60e7d-53ee-483f-9112-cdb08a396ade service nova] Lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ad65848b-cce6-4bc8-9a09-ac4d22892c0a req-08c60e7d-53ee-483f-9112-cdb08a396ade service nova] Lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG nova.compute.manager [req-ad65848b-cce6-4bc8-9a09-ac4d22892c0a req-08c60e7d-53ee-483f-9112-cdb08a396ade service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] No waiting events found dispatching network-vif-plugged-892ff129-c9ca-4295-be61-993b81b855fb {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:35:19 user nova-compute[72081]: WARNING nova.compute.manager [req-ad65848b-cce6-4bc8-9a09-ac4d22892c0a req-08c60e7d-53ee-483f-9112-cdb08a396ade service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Received unexpected event network-vif-plugged-892ff129-c9ca-4295-be61-993b81b855fb for instance with vm_state building and task_state spawning.
Apr 12 03:35:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:21 user nova-compute[72081]: DEBUG nova.compute.manager [req-8fdb76bb-39e6-4666-8686-c4660d0c67b8 req-b1bb5d54-bcff-4564-9bc0-380bd1cc2b9d service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Received event network-vif-plugged-93c27018-4ee9-42be-81a1-2b9377a689f5 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:35:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8fdb76bb-39e6-4666-8686-c4660d0c67b8 req-b1bb5d54-bcff-4564-9bc0-380bd1cc2b9d service nova] Acquiring lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:35:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8fdb76bb-39e6-4666-8686-c4660d0c67b8 req-b1bb5d54-bcff-4564-9bc0-380bd1cc2b9d service nova] Lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:35:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8fdb76bb-39e6-4666-8686-c4660d0c67b8 req-b1bb5d54-bcff-4564-9bc0-380bd1cc2b9d service nova] Lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:35:21 user nova-compute[72081]: DEBUG nova.compute.manager [req-8fdb76bb-39e6-4666-8686-c4660d0c67b8 req-b1bb5d54-bcff-4564-9bc0-380bd1cc2b9d service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] No waiting events found dispatching network-vif-plugged-93c27018-4ee9-42be-81a1-2b9377a689f5 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:35:21 user nova-compute[72081]: WARNING nova.compute.manager [req-8fdb76bb-39e6-4666-8686-c4660d0c67b8 req-b1bb5d54-bcff-4564-9bc0-380bd1cc2b9d service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Received unexpected event network-vif-plugged-93c27018-4ee9-42be-81a1-2b9377a689f5 for instance with vm_state building and task_state spawning.
Apr 12 03:35:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:21 user nova-compute[72081]: DEBUG nova.compute.manager [req-c365588f-72a9-407b-af87-fa05cec03309 req-6cbb7de6-6091-413b-8d4f-7894489a4e2c service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Received event network-vif-plugged-892ff129-c9ca-4295-be61-993b81b855fb {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:35:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c365588f-72a9-407b-af87-fa05cec03309 req-6cbb7de6-6091-413b-8d4f-7894489a4e2c service nova] Acquiring lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:35:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c365588f-72a9-407b-af87-fa05cec03309 req-6cbb7de6-6091-413b-8d4f-7894489a4e2c service nova] Lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:35:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c365588f-72a9-407b-af87-fa05cec03309 req-6cbb7de6-6091-413b-8d4f-7894489a4e2c service nova] Lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:35:21 user nova-compute[72081]: DEBUG nova.compute.manager [req-c365588f-72a9-407b-af87-fa05cec03309 req-6cbb7de6-6091-413b-8d4f-7894489a4e2c service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] No waiting events found dispatching network-vif-plugged-892ff129-c9ca-4295-be61-993b81b855fb {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:35:21 user nova-compute[72081]: WARNING nova.compute.manager [req-c365588f-72a9-407b-af87-fa05cec03309 req-6cbb7de6-6091-413b-8d4f-7894489a4e2c service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Received unexpected event network-vif-plugged-892ff129-c9ca-4295-be61-993b81b855fb for instance with vm_state building and task_state spawning.
Apr 12 03:35:22 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270522.5260794, a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:35:22 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] VM Resumed (Lifecycle Event)
Apr 12 03:35:22 user nova-compute[72081]: DEBUG nova.compute.manager [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:35:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:35:22 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Instance spawned successfully.
Apr 12 03:35:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:35:22 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:35:22 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:35:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:35:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:35:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:35:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:35:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:35:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:35:22 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:35:22 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270522.5279279, a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:35:22 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] VM Started (Lifecycle Event)
Apr 12 03:35:22 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:35:22 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:35:22 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:35:22 user nova-compute[72081]: INFO nova.compute.manager [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Took 7.09 seconds to spawn the instance on the hypervisor.
Apr 12 03:35:22 user nova-compute[72081]: DEBUG nova.compute.manager [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:35:22 user nova-compute[72081]: INFO nova.compute.manager [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Took 7.87 seconds to build instance.
Apr 12 03:35:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6bdf548d-a951-49c3-95dc-fff4d64fc7f4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.962s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:35:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:23 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270523.0617764, 49d33636-1dbb-4863-b0f3-9f1578e23e67 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:35:23 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] VM Resumed (Lifecycle Event)
Apr 12 03:35:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:35:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:35:23 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Instance spawned successfully.
Apr 12 03:35:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:35:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:35:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:35:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:35:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:35:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:35:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:35:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:35:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:35:23 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:35:23 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270523.06297, 49d33636-1dbb-4863-b0f3-9f1578e23e67 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:35:23 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] VM Started (Lifecycle Event)
Apr 12 03:35:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:35:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:35:23 user nova-compute[72081]: INFO nova.compute.manager [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Took 5.40 seconds to spawn the instance on the hypervisor.
Apr 12 03:35:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:35:23 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:35:23 user nova-compute[72081]: DEBUG nova.compute.manager [req-a16755b2-d6ee-42aa-ae00-1de4840003a8 req-cef7a226-92b1-49e2-b1fb-dc7a6cb5d19b service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Received event network-vif-plugged-93c27018-4ee9-42be-81a1-2b9377a689f5 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:35:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a16755b2-d6ee-42aa-ae00-1de4840003a8 req-cef7a226-92b1-49e2-b1fb-dc7a6cb5d19b service nova] Acquiring lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:35:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a16755b2-d6ee-42aa-ae00-1de4840003a8 req-cef7a226-92b1-49e2-b1fb-dc7a6cb5d19b service nova] Lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:35:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a16755b2-d6ee-42aa-ae00-1de4840003a8 req-cef7a226-92b1-49e2-b1fb-dc7a6cb5d19b service nova] Lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:35:23 user nova-compute[72081]: DEBUG nova.compute.manager [req-a16755b2-d6ee-42aa-ae00-1de4840003a8 req-cef7a226-92b1-49e2-b1fb-dc7a6cb5d19b service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] No waiting events found dispatching network-vif-plugged-93c27018-4ee9-42be-81a1-2b9377a689f5 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:35:23 user nova-compute[72081]: WARNING nova.compute.manager [req-a16755b2-d6ee-42aa-ae00-1de4840003a8 req-cef7a226-92b1-49e2-b1fb-dc7a6cb5d19b service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Received unexpected event network-vif-plugged-93c27018-4ee9-42be-81a1-2b9377a689f5 for instance with vm_state building and task_state spawning.
Apr 12 03:35:23 user nova-compute[72081]: INFO nova.compute.manager [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Took 17.11 seconds to build instance.
Apr 12 03:35:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5db1bca2-53ef-4cb9-a3c4-b67a05a4327c tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "49d33636-1dbb-4863-b0f3-9f1578e23e67" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 17.206s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:35:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:35:47 user nova-compute[72081]: INFO nova.compute.manager [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Detaching volume e8b7fdb9-05e3-4870-b435-4c063881640f
Apr 12 03:35:47 user nova-compute[72081]: INFO nova.virt.block_device [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Attempting to driver detach volume e8b7fdb9-05e3-4870-b435-4c063881640f from mountpoint /dev/vdc
Apr 12 03:35:47 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Attempting to detach device vdc from instance b23d4bcd-989a-45e1-89a3-6e3d30229a32 from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:35:47 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:35:47 user nova-compute[72081]:   <source dev="/dev/sdd"/>
Apr 12 03:35:47 user nova-compute[72081]:   <target dev="vdc" bus="virtio"/>
Apr 12 03:35:47 user nova-compute[72081]:   <serial>e8b7fdb9-05e3-4870-b435-4c063881640f</serial>
Apr 12 03:35:47 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Apr 12 03:35:47 user nova-compute[72081]: </disk>
Apr 12 03:35:47 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:35:47 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Successfully detached device vdc from instance b23d4bcd-989a-45e1-89a3-6e3d30229a32 from the persistent domain config.
Apr 12 03:35:47 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] (1/8): Attempting to detach device vdc with device alias virtio-disk2 from instance b23d4bcd-989a-45e1-89a3-6e3d30229a32 from the live domain config. {{(pid=72081) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:35:47 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:35:47 user nova-compute[72081]:   <source dev="/dev/sdd"/>
Apr 12 03:35:47 user nova-compute[72081]:   <target dev="vdc" bus="virtio"/>
Apr 12 03:35:47 user nova-compute[72081]:   <serial>e8b7fdb9-05e3-4870-b435-4c063881640f</serial>
Apr 12 03:35:47 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Apr 12 03:35:47 user nova-compute[72081]: </disk>
Apr 12 03:35:47 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Received event <DeviceRemovedEvent: 1681270547.6306791, b23d4bcd-989a-45e1-89a3-6e3d30229a32 => virtio-disk2> from libvirt while the driver is waiting for it; dispatched. {{(pid=72081) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Start waiting for the detach event from libvirt for device vdc with device alias virtio-disk2 for instance b23d4bcd-989a-45e1-89a3-6e3d30229a32 {{(pid=72081) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 12 03:35:47 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Successfully detached device vdc from instance b23d4bcd-989a-45e1-89a3-6e3d30229a32 from the live domain config.
Apr 12 03:35:47 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tWwylQdY', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:35:47 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:35:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[79d48dde-01c9-4231-bf58-1bf8e4e95e21]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:35:47 user nova-compute[72081]: No targets found.
Apr 12 03:35:47 user nova-compute[72081]: iSNS:
Apr 12 03:35:47 user nova-compute[72081]: No targets found.
Apr 12 03:35:47 user nova-compute[72081]: STATIC:
Apr 12 03:35:47 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:35:47 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:35:47 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:35:47 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f
Apr 12 03:35:47 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:35:47 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:35:47 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f
Apr 12 03:35:47 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:35:47 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:35:47 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939
Apr 12 03:35:47 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:35:47 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:35:47 user nova-compute[72081]: FIRMWARE:
Apr 12 03:35:47 user nova-compute[72081]: No targets found.
Apr 12 03:35:47 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:35:47 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:35:47 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[76e82eb8-4ce8-407d-a59f-4da14c2a6f2b]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[cc06184c-4062-4945-b9c9-67147ce0ff88]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:35:47 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:35:47 user nova-compute[72081]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f (non-flash)
Apr 12 03:35:47 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:35:47 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:35:47 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:35:47 user nova-compute[72081]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f (non-flash)
Apr 12 03:35:47 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:35:47 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fb6faa200>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f'): ({'sdd'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Removing single pathed devices sdd {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[50575b4a-0c8b-4f9e-9791-b792c224e843]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[dab04ce8-fbf4-4a3e-9a65-f29077d35653]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Flushing IO for device /dev/sdd {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.013s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8cfdfe59-cd44-4a9c-8048-42a3315b13b3]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.047s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0e129d40-f363-4796-bb54-5a3e622d0aab]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] SCSI volumes sdd have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[dc70be4a-ef83-4fbf-84b5-6ef956e4da1b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f -p 172.16.0.220:3260 --logout" returned: 0 in 0.051s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9b7c7f65-477f-40b6-bce3-f16d062a2e12]: (4, ('Logging out of session [sid: 16, target: iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f, portal: 172.16.0.220,3260]\nLogout of [sid: 16, target: iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 16, target: iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f, portal: 172.16.0.220,3260]
Apr 12 03:35:47 user nova-compute[72081]: Logout of [sid: 16, target: iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f, portal: 172.16.0.220,3260] successful.
Apr 12 03:35:47 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e8b7fdb9-05e3-4870-b435-4c063881640f -p 172.16.0.220:3260 --op delete" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[dda53cad-259b-48b9-9441-8552d1946b5e]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.295s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] <== disconnect_volume: return (296ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:35:47 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:35:48 user nova-compute[72081]: DEBUG nova.objects.instance [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lazy-loading 'flavor' on Instance uuid b23d4bcd-989a-45e1-89a3-6e3d30229a32 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:35:48 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-caedadc9-b7e9-4b7a-b974-585360cca50d tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.638s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:35:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:56 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:35:57 user nova-compute[72081]: DEBUG nova.compute.manager [req-240cc9fa-298c-47f3-9cf7-725932390b12 req-8ba5bf76-37da-4e99-a931-5d1fdd011b12 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Received event network-changed-b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:35:57 user nova-compute[72081]: DEBUG nova.compute.manager [req-240cc9fa-298c-47f3-9cf7-725932390b12 req-8ba5bf76-37da-4e99-a931-5d1fdd011b12 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Refreshing instance network info cache due to event network-changed-b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:35:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-240cc9fa-298c-47f3-9cf7-725932390b12 req-8ba5bf76-37da-4e99-a931-5d1fdd011b12 service nova] Acquiring lock "refresh_cache-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:35:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-240cc9fa-298c-47f3-9cf7-725932390b12 req-8ba5bf76-37da-4e99-a931-5d1fdd011b12 service nova] Acquired lock "refresh_cache-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:35:57 user nova-compute[72081]: DEBUG nova.network.neutron [req-240cc9fa-298c-47f3-9cf7-725932390b12 req-8ba5bf76-37da-4e99-a931-5d1fdd011b12 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Refreshing network info cache for port b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:35:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Acquiring lock "interface-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-None" by "nova.compute.manager.ComputeManager.attach_interface.<locals>.do_attach_interface" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:35:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "interface-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-None" acquired by "nova.compute.manager.ComputeManager.attach_interface.<locals>.do_attach_interface" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:35:58 user nova-compute[72081]: DEBUG nova.objects.instance [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lazy-loading 'flavor' on Instance uuid 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:35:58 user nova-compute[72081]: DEBUG nova.network.neutron [req-240cc9fa-298c-47f3-9cf7-725932390b12 req-8ba5bf76-37da-4e99-a931-5d1fdd011b12 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Updated VIF entry in instance network info cache for port b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:35:58 user nova-compute[72081]: DEBUG nova.network.neutron [req-240cc9fa-298c-47f3-9cf7-725932390b12 req-8ba5bf76-37da-4e99-a931-5d1fdd011b12 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Updating instance_info_cache with network_info: [{"id": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "address": "fa:16:3e:46:28:01", "network": {"id": "8d284b3a-85a8-45a3-845f-271f3fe80540", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-354475413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.20", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a327d22640aa4a7caeb35fb0ed2dccc8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb0b22dc0-c2", "ovs_interfaceid": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:35:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-240cc9fa-298c-47f3-9cf7-725932390b12 req-8ba5bf76-37da-4e99-a931-5d1fdd011b12 service nova] Releasing lock "refresh_cache-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:35:58 user nova-compute[72081]: DEBUG nova.objects.instance [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lazy-loading 'pci_requests' on Instance uuid 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:35:58 user nova-compute[72081]: DEBUG nova.network.neutron [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:35:58 user nova-compute[72081]: DEBUG nova.policy [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '3a7c2b6c06a745de95fae59f8b02c06d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a327d22640aa4a7caeb35fb0ed2dccc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:35:59 user nova-compute[72081]: DEBUG nova.network.neutron [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Successfully created port: 1fcf28a7-3fd9-4283-a0df-be0377b96d42 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:35:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:35:59 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:35:59 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG nova.network.neutron [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Successfully updated port: 1fcf28a7-3fd9-4283-a0df-be0377b96d42 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Acquiring lock "refresh_cache-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Acquired lock "refresh_cache-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG nova.network.neutron [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG nova.compute.manager [req-0498ea5f-5f3a-4e2e-a989-572ed3e7adf3 req-213a071b-2683-4c77-b48a-0a6685d8b175 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Received event network-changed-1fcf28a7-3fd9-4283-a0df-be0377b96d42 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG nova.compute.manager [req-0498ea5f-5f3a-4e2e-a989-572ed3e7adf3 req-213a071b-2683-4c77-b48a-0a6685d8b175 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Refreshing instance network info cache due to event network-changed-1fcf28a7-3fd9-4283-a0df-be0377b96d42. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0498ea5f-5f3a-4e2e-a989-572ed3e7adf3 req-213a071b-2683-4c77-b48a-0a6685d8b175 service nova] Acquiring lock "refresh_cache-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Cleaning up deleted instances with incomplete migration  {{(pid=72081) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG nova.network.neutron [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Updating instance_info_cache with network_info: [{"id": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "address": "fa:16:3e:46:28:01", "network": {"id": "8d284b3a-85a8-45a3-845f-271f3fe80540", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-354475413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.20", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a327d22640aa4a7caeb35fb0ed2dccc8", "mtu": null, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb0b22dc0-c2", "ovs_interfaceid": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "1fcf28a7-3fd9-4283-a0df-be0377b96d42", "address": "fa:16:3e:d3:72:8f", "network": {"id": "a1000896-9015-4517-a0b5-bd056cd9aa89", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-1097172279", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "a327d22640aa4a7caeb35fb0ed2dccc8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1fcf28a7-3f", "ovs_interfaceid": "1fcf28a7-3fd9-4283-a0df-be0377b96d42", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Releasing lock "refresh_cache-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0498ea5f-5f3a-4e2e-a989-572ed3e7adf3 req-213a071b-2683-4c77-b48a-0a6685d8b175 service nova] Acquired lock "refresh_cache-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG nova.network.neutron [req-0498ea5f-5f3a-4e2e-a989-572ed3e7adf3 req-213a071b-2683-4c77-b48a-0a6685d8b175 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Refreshing network info cache for port 1fcf28a7-3fd9-4283-a0df-be0377b96d42 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:34:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-934756275',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-934756275',id=15,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGqZ2JOd0oMveikCRo3egpYXw6F7M9CSnvg8uuSzQFHFmYKOQ1+3ysT+GbPl8KtibugwjCH/8obQl0aNd57dyywpuoimEfD8++mntdrmImk7chBwlJ4Kqnvt9BRQOwzT8Q==',key_name='tempest-keypair-52848813',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:34:18Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='a327d22640aa4a7caeb35fb0ed2dccc8',ramdisk_id='',reservation_id='r-autn96ne',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-1144278074',owner_user_name='tempest-TaggedAttachmentsTest-1144278074-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:34:19Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3a7c2b6c06a745de95fae59f8b02c06d',uuid=37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "1fcf28a7-3fd9-4283-a0df-be0377b96d42", "address": "fa:16:3e:d3:72:8f", "network": {"id": "a1000896-9015-4517-a0b5-bd056cd9aa89", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-1097172279", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "a327d22640aa4a7caeb35fb0ed2dccc8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1fcf28a7-3f", "ovs_interfaceid": "1fcf28a7-3fd9-4283-a0df-be0377b96d42", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Converting VIF {"id": "1fcf28a7-3fd9-4283-a0df-be0377b96d42", "address": "fa:16:3e:d3:72:8f", "network": {"id": "a1000896-9015-4517-a0b5-bd056cd9aa89", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-1097172279", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "a327d22640aa4a7caeb35fb0ed2dccc8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1fcf28a7-3f", "ovs_interfaceid": "1fcf28a7-3fd9-4283-a0df-be0377b96d42", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d3:72:8f,bridge_name='br-int',has_traffic_filtering=True,id=1fcf28a7-3fd9-4283-a0df-be0377b96d42,network=Network(a1000896-9015-4517-a0b5-bd056cd9aa89),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1fcf28a7-3f') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG os_vif [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:d3:72:8f,bridge_name='br-int',has_traffic_filtering=True,id=1fcf28a7-3fd9-4283-a0df-be0377b96d42,network=Network(a1000896-9015-4517-a0b5-bd056cd9aa89),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1fcf28a7-3f') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap1fcf28a7-3f, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap1fcf28a7-3f, col_values=(('external_ids', {'iface-id': '1fcf28a7-3fd9-4283-a0df-be0377b96d42', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:d3:72:8f', 'vm-uuid': '37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:00 user nova-compute[72081]: INFO os_vif [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:d3:72:8f,bridge_name='br-int',has_traffic_filtering=True,id=1fcf28a7-3fd9-4283-a0df-be0377b96d42,network=Network(a1000896-9015-4517-a0b5-bd056cd9aa89),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1fcf28a7-3f')
Apr 12 03:36:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:34:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-934756275',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-934756275',id=15,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGqZ2JOd0oMveikCRo3egpYXw6F7M9CSnvg8uuSzQFHFmYKOQ1+3ysT+GbPl8KtibugwjCH/8obQl0aNd57dyywpuoimEfD8++mntdrmImk7chBwlJ4Kqnvt9BRQOwzT8Q==',key_name='tempest-keypair-52848813',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:34:18Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='a327d22640aa4a7caeb35fb0ed2dccc8',ramdisk_id='',reservation_id='r-autn96ne',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-1144278074',owner_user_name='tempest-TaggedAttachmentsTest-1144278074-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:34:19Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3a7c2b6c06a745de95fae59f8b02c06d',uuid=37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "1fcf28a7-3fd9-4283-a0df-be0377b96d42", "address": "fa:16:3e:d3:72:8f", "network": {"id": "a1000896-9015-4517-a0b5-bd056cd9aa89", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-1097172279", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "a327d22640aa4a7caeb35fb0ed2dccc8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1fcf28a7-3f", "ovs_interfaceid": "1fcf28a7-3fd9-4283-a0df-be0377b96d42", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Converting VIF {"id": "1fcf28a7-3fd9-4283-a0df-be0377b96d42", "address": "fa:16:3e:d3:72:8f", "network": {"id": "a1000896-9015-4517-a0b5-bd056cd9aa89", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-1097172279", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "a327d22640aa4a7caeb35fb0ed2dccc8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1fcf28a7-3f", "ovs_interfaceid": "1fcf28a7-3fd9-4283-a0df-be0377b96d42", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d3:72:8f,bridge_name='br-int',has_traffic_filtering=True,id=1fcf28a7-3fd9-4283-a0df-be0377b96d42,network=Network(a1000896-9015-4517-a0b5-bd056cd9aa89),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1fcf28a7-3f') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] attach device xml: <interface type="ethernet">
Apr 12 03:36:00 user nova-compute[72081]:   <mac address="fa:16:3e:d3:72:8f"/>
Apr 12 03:36:00 user nova-compute[72081]:   <model type="virtio"/>
Apr 12 03:36:00 user nova-compute[72081]:   <mtu size="1442"/>
Apr 12 03:36:00 user nova-compute[72081]:   <target dev="tap1fcf28a7-3f"/>
Apr 12 03:36:00 user nova-compute[72081]: </interface>
Apr 12 03:36:00 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] No BDM found with device name hda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] No VIF found with MAC fa:16:3e:46:28:01, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] set metadata xml: <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:36:01 user nova-compute[72081]:   <nova:package version="0.0.0"/>
Apr 12 03:36:01 user nova-compute[72081]:   <nova:name>tempest-device-tagging-server-934756275</nova:name>
Apr 12 03:36:01 user nova-compute[72081]:   <nova:creationTime>2023-04-12 03:36:01</nova:creationTime>
Apr 12 03:36:01 user nova-compute[72081]:   <nova:flavor name="m1.nano">
Apr 12 03:36:01 user nova-compute[72081]:     <nova:memory>128</nova:memory>
Apr 12 03:36:01 user nova-compute[72081]:     <nova:disk>1</nova:disk>
Apr 12 03:36:01 user nova-compute[72081]:     <nova:swap>0</nova:swap>
Apr 12 03:36:01 user nova-compute[72081]:     <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:36:01 user nova-compute[72081]:     <nova:vcpus>1</nova:vcpus>
Apr 12 03:36:01 user nova-compute[72081]:   </nova:flavor>
Apr 12 03:36:01 user nova-compute[72081]:   <nova:owner>
Apr 12 03:36:01 user nova-compute[72081]:     <nova:user uuid="3a7c2b6c06a745de95fae59f8b02c06d">tempest-TaggedAttachmentsTest-1144278074-project-member</nova:user>
Apr 12 03:36:01 user nova-compute[72081]:     <nova:project uuid="a327d22640aa4a7caeb35fb0ed2dccc8">tempest-TaggedAttachmentsTest-1144278074</nova:project>
Apr 12 03:36:01 user nova-compute[72081]:   </nova:owner>
Apr 12 03:36:01 user nova-compute[72081]:   <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:36:01 user nova-compute[72081]:   <nova:ports>
Apr 12 03:36:01 user nova-compute[72081]:     <nova:port uuid="b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958">
Apr 12 03:36:01 user nova-compute[72081]:       <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Apr 12 03:36:01 user nova-compute[72081]:     </nova:port>
Apr 12 03:36:01 user nova-compute[72081]:     <nova:port uuid="1fcf28a7-3fd9-4283-a0df-be0377b96d42">
Apr 12 03:36:01 user nova-compute[72081]:       <nova:ip type="fixed" address="10.10.10.34" ipVersion="4"/>
Apr 12 03:36:01 user nova-compute[72081]:     </nova:port>
Apr 12 03:36:01 user nova-compute[72081]:   </nova:ports>
Apr 12 03:36:01 user nova-compute[72081]: </nova:instance>
Apr 12 03:36:01 user nova-compute[72081]:  {{(pid=72081) set_metadata /opt/stack/nova/nova/virt/libvirt/guest.py:359}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f6e04353-87b6-4ef2-a095-9bb54e18b617 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "interface-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-None" "released" by "nova.compute.manager.ComputeManager.attach_interface.<locals>.do_attach_interface" :: held 3.133s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Acquiring lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG nova.objects.instance [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lazy-loading 'flavor' on Instance uuid 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG nova.network.neutron [req-0498ea5f-5f3a-4e2e-a989-572ed3e7adf3 req-213a071b-2683-4c77-b48a-0a6685d8b175 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Updated VIF entry in instance network info cache for port 1fcf28a7-3fd9-4283-a0df-be0377b96d42. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG nova.network.neutron [req-0498ea5f-5f3a-4e2e-a989-572ed3e7adf3 req-213a071b-2683-4c77-b48a-0a6685d8b175 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Updating instance_info_cache with network_info: [{"id": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "address": "fa:16:3e:46:28:01", "network": {"id": "8d284b3a-85a8-45a3-845f-271f3fe80540", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-354475413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.20", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a327d22640aa4a7caeb35fb0ed2dccc8", "mtu": null, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb0b22dc0-c2", "ovs_interfaceid": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "1fcf28a7-3fd9-4283-a0df-be0377b96d42", "address": "fa:16:3e:d3:72:8f", "network": {"id": "a1000896-9015-4517-a0b5-bd056cd9aa89", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-1097172279", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "a327d22640aa4a7caeb35fb0ed2dccc8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1fcf28a7-3f", "ovs_interfaceid": "1fcf28a7-3fd9-4283-a0df-be0377b96d42", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0498ea5f-5f3a-4e2e-a989-572ed3e7adf3 req-213a071b-2683-4c77-b48a-0a6685d8b175 service nova] Releasing lock "refresh_cache-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.067s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Acquiring lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:01 user nova-compute[72081]: INFO nova.compute.manager [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Attaching volume 8c45cb5c-d38a-4f37-959c-08611a51e9a8 to /dev/vdb
Apr 12 03:36:01 user nova-compute[72081]: DEBUG os_brick.utils [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f7ce83ef-c068-4fb8-88ff-2260f3e6c44b]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b587db07-b710-44a1-9b32-e7bcead8b15b]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[63c5813a-5231-48f8-89fc-4a4f8e1255ea]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1a9cf39c-dad6-410d-a915-58a1650df9c3]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:36:01 user nova-compute[72081]: INFO nova.compute.manager [None req-c25d0d32-9025-4b76-abf2-a42652d8745b tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Unrescuing
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c25d0d32-9025-4b76-abf2-a42652d8745b tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Acquiring lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c25d0d32-9025-4b76-abf2-a42652d8745b tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Acquired lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG nova.network.neutron [None req-c25d0d32-9025-4b76-abf2-a42652d8745b tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:36:01 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5a5713e7-6772-4348-b9a9-8b228f92d3fc]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG os_brick.utils [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] <== get_connector_properties: return (132ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Updating existing volume attachment record: 1a48db75-8d16-465a-99e7-88191dc27737 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:36:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG nova.compute.manager [req-fd9e0537-1001-4557-941f-d06a2ebceea8 req-5877be66-acb7-432a-b549-3bbc3de97ae2 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Received event network-vif-plugged-1fcf28a7-3fd9-4283-a0df-be0377b96d42 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-fd9e0537-1001-4557-941f-d06a2ebceea8 req-5877be66-acb7-432a-b549-3bbc3de97ae2 service nova] Acquiring lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-fd9e0537-1001-4557-941f-d06a2ebceea8 req-5877be66-acb7-432a-b549-3bbc3de97ae2 service nova] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-fd9e0537-1001-4557-941f-d06a2ebceea8 req-5877be66-acb7-432a-b549-3bbc3de97ae2 service nova] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG nova.compute.manager [req-fd9e0537-1001-4557-941f-d06a2ebceea8 req-5877be66-acb7-432a-b549-3bbc3de97ae2 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] No waiting events found dispatching network-vif-plugged-1fcf28a7-3fd9-4283-a0df-be0377b96d42 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:02 user nova-compute[72081]: WARNING nova.compute.manager [req-fd9e0537-1001-4557-941f-d06a2ebceea8 req-5877be66-acb7-432a-b549-3bbc3de97ae2 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Received unexpected event network-vif-plugged-1fcf28a7-3fd9-4283-a0df-be0377b96d42 for instance with vm_state active and task_state None.
Apr 12 03:36:02 user nova-compute[72081]: DEBUG nova.compute.manager [req-fd9e0537-1001-4557-941f-d06a2ebceea8 req-5877be66-acb7-432a-b549-3bbc3de97ae2 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Received event network-vif-plugged-1fcf28a7-3fd9-4283-a0df-be0377b96d42 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-fd9e0537-1001-4557-941f-d06a2ebceea8 req-5877be66-acb7-432a-b549-3bbc3de97ae2 service nova] Acquiring lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-fd9e0537-1001-4557-941f-d06a2ebceea8 req-5877be66-acb7-432a-b549-3bbc3de97ae2 service nova] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-fd9e0537-1001-4557-941f-d06a2ebceea8 req-5877be66-acb7-432a-b549-3bbc3de97ae2 service nova] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG nova.compute.manager [req-fd9e0537-1001-4557-941f-d06a2ebceea8 req-5877be66-acb7-432a-b549-3bbc3de97ae2 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] No waiting events found dispatching network-vif-plugged-1fcf28a7-3fd9-4283-a0df-be0377b96d42 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:02 user nova-compute[72081]: WARNING nova.compute.manager [req-fd9e0537-1001-4557-941f-d06a2ebceea8 req-5877be66-acb7-432a-b549-3bbc3de97ae2 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Received unexpected event network-vif-plugged-1fcf28a7-3fd9-4283-a0df-be0377b96d42 for instance with vm_state active and task_state None.
Apr 12 03:36:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG nova.network.neutron [None req-c25d0d32-9025-4b76-abf2-a42652d8745b tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Updating instance_info_cache with network_info: [{"id": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "address": "fa:16:3e:c0:f6:ee", "network": {"id": "b4464ad0-6022-43e7-bfe4-229e6a3a0600", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1704232678-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.188", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a027d7708e6747fe970c388d6ba66359", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcff588d4-f4", "ovs_interfaceid": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c25d0d32-9025-4b76-abf2-a42652d8745b tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Releasing lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG nova.objects.instance [None req-c25d0d32-9025-4b76-abf2-a42652d8745b tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lazy-loading 'flavor' on Instance uuid 72ba1218-065d-40c4-8503-7c4155e47503 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json" returned: 0 in 0.177s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-ce265b5c-0dd8-4ef7-bd5c-281144592a24 req-4d355a7c-d1f9-408a-aa96-5dc954603230 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received event network-changed-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-ce265b5c-0dd8-4ef7-bd5c-281144592a24 req-4d355a7c-d1f9-408a-aa96-5dc954603230 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Refreshing instance network info cache due to event network-changed-cff588d4-f46a-4d44-a3e6-1c04f6abd380. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ce265b5c-0dd8-4ef7-bd5c-281144592a24 req-4d355a7c-d1f9-408a-aa96-5dc954603230 service nova] Acquiring lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ce265b5c-0dd8-4ef7-bd5c-281144592a24 req-4d355a7c-d1f9-408a-aa96-5dc954603230 service nova] Acquired lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG nova.network.neutron [req-ce265b5c-0dd8-4ef7-bd5c-281144592a24 req-4d355a7c-d1f9-408a-aa96-5dc954603230 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Refreshing network info cache for port cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG nova.objects.instance [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lazy-loading 'flavor' on Instance uuid 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:36:03 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Ignoring supplied device name: /dev/vdb
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.065s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.158s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:03 user nova-compute[72081]: INFO nova.compute.manager [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Attaching volume b48ff4ab-9969-4337-bce5-6dcdfced462a to /dev/vdb
Apr 12 03:36:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG os_brick.utils [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2b6481b4-e235-479d-8033-e4c9d5c515fc]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7f8ac113-5173-4ebd-9c5f-47539f8b88e7]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[59121f6f-f14f-4940-8997-aa22c642d629]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b18f3a0a-d3a2-4caf-b78c-ea9ac922ef8a]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:36:03 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[47a004b2-24f8-47f7-9816-818a1e227730]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG os_brick.utils [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] <== get_connector_properties: return (173ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Updating existing volume attachment record: e0bdb88c-d143-44b9-a8d9-a97f09d530f3 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:36:03 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Instance destroyed successfully.
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json" returned: 0 in 0.154s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG nova.network.neutron [req-ce265b5c-0dd8-4ef7-bd5c-281144592a24 req-4d355a7c-d1f9-408a-aa96-5dc954603230 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Updated VIF entry in instance network info cache for port cff588d4-f46a-4d44-a3e6-1c04f6abd380. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG nova.network.neutron [req-ce265b5c-0dd8-4ef7-bd5c-281144592a24 req-4d355a7c-d1f9-408a-aa96-5dc954603230 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Updating instance_info_cache with network_info: [{"id": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "address": "fa:16:3e:c0:f6:ee", "network": {"id": "b4464ad0-6022-43e7-bfe4-229e6a3a0600", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1704232678-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.188", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a027d7708e6747fe970c388d6ba66359", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcff588d4-f4", "ovs_interfaceid": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ce265b5c-0dd8-4ef7-bd5c-281144592a24 req-4d355a7c-d1f9-408a-aa96-5dc954603230 service nova] Releasing lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk.rescue --force-share --output=json" returned: 0 in 0.145s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk.rescue --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:36:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-05a1e7d8-a1a0-49c6-b937-94afeb2c5b9c req-637e4026-2221-4b66-af0c-1845c50c32d9 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received event network-vif-unplugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-05a1e7d8-a1a0-49c6-b937-94afeb2c5b9c req-637e4026-2221-4b66-af0c-1845c50c32d9 service nova] Acquiring lock "72ba1218-065d-40c4-8503-7c4155e47503-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-05a1e7d8-a1a0-49c6-b937-94afeb2c5b9c req-637e4026-2221-4b66-af0c-1845c50c32d9 service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-05a1e7d8-a1a0-49c6-b937-94afeb2c5b9c req-637e4026-2221-4b66-af0c-1845c50c32d9 service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-05a1e7d8-a1a0-49c6-b937-94afeb2c5b9c req-637e4026-2221-4b66-af0c-1845c50c32d9 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] No waiting events found dispatching network-vif-unplugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:04 user nova-compute[72081]: WARNING nova.compute.manager [req-05a1e7d8-a1a0-49c6-b937-94afeb2c5b9c req-637e4026-2221-4b66-af0c-1845c50c32d9 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received unexpected event network-vif-unplugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 for instance with vm_state rescued and task_state unrescuing.
Apr 12 03:36:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-05a1e7d8-a1a0-49c6-b937-94afeb2c5b9c req-637e4026-2221-4b66-af0c-1845c50c32d9 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received event network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-05a1e7d8-a1a0-49c6-b937-94afeb2c5b9c req-637e4026-2221-4b66-af0c-1845c50c32d9 service nova] Acquiring lock "72ba1218-065d-40c4-8503-7c4155e47503-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-05a1e7d8-a1a0-49c6-b937-94afeb2c5b9c req-637e4026-2221-4b66-af0c-1845c50c32d9 service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-05a1e7d8-a1a0-49c6-b937-94afeb2c5b9c req-637e4026-2221-4b66-af0c-1845c50c32d9 service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-05a1e7d8-a1a0-49c6-b937-94afeb2c5b9c req-637e4026-2221-4b66-af0c-1845c50c32d9 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] No waiting events found dispatching network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:04 user nova-compute[72081]: WARNING nova.compute.manager [req-05a1e7d8-a1a0-49c6-b937-94afeb2c5b9c req-637e4026-2221-4b66-af0c-1845c50c32d9 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received unexpected event network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 for instance with vm_state rescued and task_state unrescuing.
Apr 12 03:36:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk --force-share --output=json" returned: 0 in 0.156s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:36:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bcd267d2-5b2c-4ed2-b634-0210fae94dfd/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bcd267d2-5b2c-4ed2-b634-0210fae94dfd/disk.rescue --force-share --output=json" returned: 0 in 0.144s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bcd267d2-5b2c-4ed2-b634-0210fae94dfd/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bcd267d2-5b2c-4ed2-b634-0210fae94dfd/disk.rescue --force-share --output=json" returned: 0 in 0.131s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk --force-share --output=json" returned: 0 in 0.165s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:05 user nova-compute[72081]: DEBUG nova.compute.manager [req-b6423fb7-5361-42df-ba98-fd2312027e17 req-e3e58ceb-f69b-4cb9-af0d-73190278fca3 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received event network-changed-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:05 user nova-compute[72081]: DEBUG nova.compute.manager [req-b6423fb7-5361-42df-ba98-fd2312027e17 req-e3e58ceb-f69b-4cb9-af0d-73190278fca3 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Refreshing instance network info cache due to event network-changed-cff588d4-f46a-4d44-a3e6-1c04f6abd380. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:36:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b6423fb7-5361-42df-ba98-fd2312027e17 req-e3e58ceb-f69b-4cb9-af0d-73190278fca3 service nova] Acquiring lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:36:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b6423fb7-5361-42df-ba98-fd2312027e17 req-e3e58ceb-f69b-4cb9-af0d-73190278fca3 service nova] Acquired lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:36:05 user nova-compute[72081]: DEBUG nova.network.neutron [req-b6423fb7-5361-42df-ba98-fd2312027e17 req-e3e58ceb-f69b-4cb9-af0d-73190278fca3 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Refreshing network info cache for port cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:36:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json" returned: 0 in 0.483s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG nova.network.neutron [req-b6423fb7-5361-42df-ba98-fd2312027e17 req-e3e58ceb-f69b-4cb9-af0d-73190278fca3 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Updated VIF entry in instance network info cache for port cff588d4-f46a-4d44-a3e6-1c04f6abd380. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG nova.network.neutron [req-b6423fb7-5361-42df-ba98-fd2312027e17 req-e3e58ceb-f69b-4cb9-af0d-73190278fca3 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Updating instance_info_cache with network_info: [{"id": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "address": "fa:16:3e:c0:f6:ee", "network": {"id": "b4464ad0-6022-43e7-bfe4-229e6a3a0600", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1704232678-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.188", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a027d7708e6747fe970c388d6ba66359", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcff588d4-f4", "ovs_interfaceid": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-c25d0d32-9025-4b76-abf2-a42652d8745b tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-1b278d54-1f6a-4739-b4b3-fb8d37bee423 req-ba1028eb-0eab-4119-9d26-39a6023eb619 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received event network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b278d54-1f6a-4739-b4b3-fb8d37bee423 req-ba1028eb-0eab-4119-9d26-39a6023eb619 service nova] Acquiring lock "72ba1218-065d-40c4-8503-7c4155e47503-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b278d54-1f6a-4739-b4b3-fb8d37bee423 req-ba1028eb-0eab-4119-9d26-39a6023eb619 service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b278d54-1f6a-4739-b4b3-fb8d37bee423 req-ba1028eb-0eab-4119-9d26-39a6023eb619 service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-1b278d54-1f6a-4739-b4b3-fb8d37bee423 req-ba1028eb-0eab-4119-9d26-39a6023eb619 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] No waiting events found dispatching network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:07 user nova-compute[72081]: WARNING nova.compute.manager [req-1b278d54-1f6a-4739-b4b3-fb8d37bee423 req-ba1028eb-0eab-4119-9d26-39a6023eb619 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received unexpected event network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 for instance with vm_state rescued and task_state unrescuing.
Apr 12 03:36:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-1b278d54-1f6a-4739-b4b3-fb8d37bee423 req-ba1028eb-0eab-4119-9d26-39a6023eb619 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received event network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b278d54-1f6a-4739-b4b3-fb8d37bee423 req-ba1028eb-0eab-4119-9d26-39a6023eb619 service nova] Acquiring lock "72ba1218-065d-40c4-8503-7c4155e47503-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b278d54-1f6a-4739-b4b3-fb8d37bee423 req-ba1028eb-0eab-4119-9d26-39a6023eb619 service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b278d54-1f6a-4739-b4b3-fb8d37bee423 req-ba1028eb-0eab-4119-9d26-39a6023eb619 service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-1b278d54-1f6a-4739-b4b3-fb8d37bee423 req-ba1028eb-0eab-4119-9d26-39a6023eb619 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] No waiting events found dispatching network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:07 user nova-compute[72081]: WARNING nova.compute.manager [req-1b278d54-1f6a-4739-b4b3-fb8d37bee423 req-ba1028eb-0eab-4119-9d26-39a6023eb619 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received unexpected event network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 for instance with vm_state rescued and task_state unrescuing.
Apr 12 03:36:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Removed pending event for 72ba1218-065d-40c4-8503-7c4155e47503 due to event {{(pid=72081) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270567.198022, 72ba1218-065d-40c4-8503-7c4155e47503 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:36:07 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] VM Resumed (Lifecycle Event)
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b6423fb7-5361-42df-ba98-fd2312027e17 req-e3e58ceb-f69b-4cb9-af0d-73190278fca3 service nova] Releasing lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'fJWXBzIF', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:36:07 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:36:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:36:07 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:36:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:36:07 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:36:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:07 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:36:07 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:36:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=7448MB free_disk=26.054763793945312GB free_vcpus=1 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 -p 172.16.0.220:3260" returned: 21 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[fd60c595-008d-4b0d-a538-41c2f6fe3162]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:36:07 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:07 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] During sync_power_state the instance has a pending task (unrescuing). Skip.
Apr 12 03:36:07 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270567.19889, 72ba1218-065d-40c4-8503-7c4155e47503 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:36:07 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] VM Started (Lifecycle Event)
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5cf3178c-6bcc-4748-957c-d44a18d1c9f3]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8] added
Apr 12 03:36:07 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[96817fbb-24d2-4ad6-acd3-5d0dc6b0e48d]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[dce19a49-ef5b-4efc-bee5-9c715e63d701]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v fJWXBzIF {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v fJWXBzIF" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[525a71af-c4da-4ede-a309-2747779974d6]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'fJWXBzIF'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[742e2c6b-fd0b-45a5-a90b-c580324c4924]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[dff3f613-ca79-43e5-b9e7-1732d99a181a]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:07 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:36:07 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:07 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:07 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:36:07 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:07 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 -p 172.16.0.220:3260 --login" returned: 0 in 0.067s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe86307d-ddb6-49a8-a992-e77b85b2a7d0]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8, portal: 172.16.0.220,3260]
Apr 12 03:36:07 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8, portal: 172.16.0.220,3260] successful.
Apr 12 03:36:07 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9be07929-ff6a-4544-82fd-a704bb221b46]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[572c55d8-f5d6-4ac0-9e88-f827181d2037]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)\ntcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:07 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:36:07 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:07 user nova-compute[72081]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 (non-flash)
Apr 12 03:36:07 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:07 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:36:07 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:07 user nova-compute[72081]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 (non-flash)
Apr 12 03:36:07 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.313s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] HCTL ('35', '-', '-', 0) found on session 18 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ee01758d-e28d-4e39-9c78-980db4c71ca3]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Searching for a device in session 18 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'fJWXBzIF', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:36:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ulsJFnqz', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:36:07 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:36:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 56834002-7593-4592-a86d-23bb90d00be0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 2d0904c3-74e0-47bb-a832-758a7035b2f5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 72ba1218-065d-40c4-8503-7c4155e47503 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance bcd267d2-5b2c-4ed2-b634-0210fae94dfd actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 48977eb2-ce43-4577-a1cb-340e35703dbd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance b23d4bcd-989a-45e1-89a3-6e3d30229a32 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 23367b24-0a3b-454d-901d-8a3e5396333a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 49d33636-1dbb-4863-b0f3-9f1578e23e67 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 11 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=1920MB phys_disk=40GB used_disk=9GB total_vcpus=12 used_vcpus=11 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Refreshing inventories for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Updating ProviderTree inventory for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Updating inventory in ProviderTree for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Refreshing aggregate associations for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6, aggregates: None {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Refreshing trait associations for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6, traits: COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_USB,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_RESCUE_BFV,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE41,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_STORAGE_BUS_FDC,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_STORAGE_BUS_IDE,COMPUTE_TRUSTED_CERTS,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_GRAPHICS_MODEL_CIRRUS,HW_CPU_X86_SSSE3,HW_CPU_X86_SSE,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_EXTEND,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_STORAGE_BUS_SATA,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_GRAPHICS_MODEL_QXL,HW_CPU_X86_SSE42,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_ACCELERATORS,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,HW_CPU_X86_MMX,COMPUTE_NODE,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_DEVICE_TAGGING,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_VIOMMU_MODEL_AUTO {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.889s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Cleaning up deleted instances {{(pid=72081) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] There are 0 instances to clean {{(pid=72081) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.356s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] <== connect_volume: return (1358ms) {'type': 'block', 'scsi_wwn': '23134623230633966', 'path': '/dev/sdd'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23134623230633966', 'path': '/dev/sdd'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.997s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:36:08 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:36:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:36:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:36:08 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:36:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG nova.objects.instance [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lazy-loading 'flavor' on Instance uuid 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[72f8a2de-2615-4958-b44c-427701e28dc8]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:36:09 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] attach device xml: <disk type="block" device="disk">
Apr 12 03:36:09 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:36:09 user nova-compute[72081]:   <source dev="/dev/sdd"/>
Apr 12 03:36:09 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:36:09 user nova-compute[72081]:   <serial>8c45cb5c-d38a-4f37-959c-08611a51e9a8</serial>
Apr 12 03:36:09 user nova-compute[72081]: </disk>
Apr 12 03:36:09 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f0f69eca-a44d-4894-80ff-3b23dc8795b4]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a] added
Apr 12 03:36:09 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3f32d532-62af-4cee-befa-fa1841dad3aa]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[477da045-19aa-4a4a-b1f7-7452cca9976d]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a -p 172.16.0.220:3260 --op update -n node.session.auth.username -v ulsJFnqz {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a -p 172.16.0.220:3260 --op update -n node.session.auth.username -v ulsJFnqz" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f2e2d28b-7743-49d7-b2a0-fca18e874a49]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'ulsJFnqz'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c8a684a2-05e2-46a7-b340-cad0380f939f]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[28e74359-d872-4020-975d-d9af6624ec59]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)\ntcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:09 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:36:09 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:09 user nova-compute[72081]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 (non-flash)
Apr 12 03:36:09 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:09 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:36:09 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:09 user nova-compute[72081]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 (non-flash)
Apr 12 03:36:09 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a -p 172.16.0.220:3260 --login" returned: 0 in 0.077s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[cdf94f7c-b91e-4b58-a1fb-45c8be9dacc9]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a, portal: 172.16.0.220,3260]
Apr 12 03:36:09 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a, portal: 172.16.0.220,3260] successful.
Apr 12 03:36:09 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c6b00eb5-f976-41c1-909a-3c1921f59c50]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[21a4e6c9-ab7c-4e0e-a057-698ec6a23cd5]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)\ntcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:09 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:36:09 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:09 user nova-compute[72081]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 (non-flash)
Apr 12 03:36:09 user nova-compute[72081]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a (non-flash)
Apr 12 03:36:09 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:09 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:36:09 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:09 user nova-compute[72081]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 (non-flash)
Apr 12 03:36:09 user nova-compute[72081]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a (non-flash)
Apr 12 03:36:09 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.325s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] HCTL ('37', '-', '-', 0) found on session 19 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Scanning host 37 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5547ce52-fe20-4720-8c73-2286196583f1]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Searching for a device in session 19 and hctl ['37', '*', '*', 0] yield: sdf {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Connected to sdf using {'target_iqn': 'iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ulsJFnqz', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] No BDM found with device name hda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] No VIF found with MAC fa:16:3e:46:28:01, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-2d0904c3-74e0-47bb-a832-758a7035b2f5" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-2d0904c3-74e0-47bb-a832-758a7035b2f5" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 03:36:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1ec8a6fa-9f33-4cf4-b3bc-86bb15f317e5 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 7.879s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:10 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Updating instance_info_cache with network_info: [{"id": "dce84390-a2f8-4047-b513-eec5845a2399", "address": "fa:16:3e:6e:2e:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdce84390-a2", "ovs_interfaceid": "dce84390-a2f8-4047-b513-eec5845a2399", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:36:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-2d0904c3-74e0-47bb-a832-758a7035b2f5" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:36:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 03:36:10 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:36:10 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:36:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:36:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.362s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:36:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] <== connect_volume: return (2359ms) {'type': 'block', 'scsi_wwn': '26261313236303638', 'path': '/dev/sdf'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:36:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26261313236303638', 'path': '/dev/sdf'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:36:10 user nova-compute[72081]: DEBUG nova.objects.instance [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lazy-loading 'flavor' on Instance uuid 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:36:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] attach device xml: <disk type="block" device="disk">
Apr 12 03:36:10 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:36:10 user nova-compute[72081]:   <source dev="/dev/sdf"/>
Apr 12 03:36:10 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:36:10 user nova-compute[72081]:   <serial>b48ff4ab-9969-4337-bce5-6dcdfced462a</serial>
Apr 12 03:36:10 user nova-compute[72081]: </disk>
Apr 12 03:36:10 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 03:36:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:36:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:36:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] No VIF found with MAC fa:16:3e:71:14:95, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:36:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Acquiring lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:11 user nova-compute[72081]: INFO nova.compute.manager [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Detaching volume 8c45cb5c-d38a-4f37-959c-08611a51e9a8
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c1862e47-d8d8-4189-9a90-94c5e8cb422f tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 7.741s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:11 user nova-compute[72081]: INFO nova.virt.block_device [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Attempting to driver detach volume 8c45cb5c-d38a-4f37-959c-08611a51e9a8 from mountpoint /dev/vdb
Apr 12 03:36:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Attempting to detach device vdb from instance 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:36:11 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:36:11 user nova-compute[72081]:   <source dev="/dev/sdd"/>
Apr 12 03:36:11 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:36:11 user nova-compute[72081]:   <serial>8c45cb5c-d38a-4f37-959c-08611a51e9a8</serial>
Apr 12 03:36:11 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
Apr 12 03:36:11 user nova-compute[72081]: </disk>
Apr 12 03:36:11 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:36:11 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Successfully detached device vdb from instance 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 from the persistent domain config.
Apr 12 03:36:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 from the live domain config. {{(pid=72081) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:36:11 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:36:11 user nova-compute[72081]:   <source dev="/dev/sdd"/>
Apr 12 03:36:11 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:36:11 user nova-compute[72081]:   <serial>8c45cb5c-d38a-4f37-959c-08611a51e9a8</serial>
Apr 12 03:36:11 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
Apr 12 03:36:11 user nova-compute[72081]: </disk>
Apr 12 03:36:11 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Received event <DeviceRemovedEvent: 1681270571.5952334, 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72081) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 {{(pid=72081) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 12 03:36:11 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Successfully detached device vdb from instance 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 from the live domain config.
Apr 12 03:36:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'fJWXBzIF', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:36:11 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:36:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3b284ea4-6a6d-4903-905a-2303f4a8d58c]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:36:11 user nova-compute[72081]: No targets found.
Apr 12 03:36:11 user nova-compute[72081]: iSNS:
Apr 12 03:36:11 user nova-compute[72081]: No targets found.
Apr 12 03:36:11 user nova-compute[72081]: STATIC:
Apr 12 03:36:11 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a
Apr 12 03:36:11 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:36:11 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:36:11 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:36:11 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:36:11 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:36:11 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f
Apr 12 03:36:11 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:36:11 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:36:11 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8
Apr 12 03:36:11 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:36:11 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:36:11 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939
Apr 12 03:36:11 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:36:11 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:36:11 user nova-compute[72081]: FIRMWARE:
Apr 12 03:36:11 user nova-compute[72081]: No targets found.
Apr 12 03:36:11 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:36:11 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:36:11 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d455b505-dac7-4b69-b9f5-663f82beaa3b]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3f949145-936c-4adc-b342-1b377b986421]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)\ntcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:11 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:36:11 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:11 user nova-compute[72081]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 (non-flash)
Apr 12 03:36:11 user nova-compute[72081]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a (non-flash)
Apr 12 03:36:11 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:11 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:36:11 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:11 user nova-compute[72081]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 (non-flash)
Apr 12 03:36:11 user nova-compute[72081]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a (non-flash)
Apr 12 03:36:11 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fba3b6710>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8'): ({'sdd'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Removing single pathed devices sdd {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7786ccc5-e76d-475d-a183-fd7777146cf7]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e066e1df-b8d0-4d0d-b663-aa9b3ade94ae]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Flushing IO for device /dev/sdd {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.012s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d25cb742-8e17-47f5-9391-31deaf9aa372]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.041s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[061ec915-2f1f-4a83-a266-f3989932d58e]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] SCSI volumes sdd have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[19f7a34f-e804-4c6a-a8e4-3049de396a72]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 -p 172.16.0.220:3260 --logout" returned: 0 in 0.056s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5de0155c-8d69-4876-8614-46da5f9aa8ea]: (4, ('Logging out of session [sid: 18, target: iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8, portal: 172.16.0.220,3260]\nLogout of [sid: 18, target: iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 18, target: iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8, portal: 172.16.0.220,3260]
Apr 12 03:36:11 user nova-compute[72081]: Logout of [sid: 18, target: iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8, portal: 172.16.0.220,3260] successful.
Apr 12 03:36:11 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8c45cb5c-d38a-4f37-959c-08611a51e9a8 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f1839cc4-ee5c-4ed6-a1f7-cdd84e31a20b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.299s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] <== disconnect_volume: return (301ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:36:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:36:12 user nova-compute[72081]: DEBUG nova.objects.instance [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lazy-loading 'flavor' on Instance uuid 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:36:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-86e4dfa8-0d38-4157-9145-55bf3844c597 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.757s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Acquiring lock "interface-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-1fcf28a7-3fd9-4283-a0df-be0377b96d42" by "nova.compute.manager.ComputeManager.detach_interface.<locals>.do_detach_interface" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "interface-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-1fcf28a7-3fd9-4283-a0df-be0377b96d42" acquired by "nova.compute.manager.ComputeManager.detach_interface.<locals>.do_detach_interface" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG nova.objects.instance [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lazy-loading 'flavor' on Instance uuid 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:34:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=InstanceDeviceMetadata,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-934756275',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-934756275',id=15,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGqZ2JOd0oMveikCRo3egpYXw6F7M9CSnvg8uuSzQFHFmYKOQ1+3ysT+GbPl8KtibugwjCH/8obQl0aNd57dyywpuoimEfD8++mntdrmImk7chBwlJ4Kqnvt9BRQOwzT8Q==',key_name='tempest-keypair-52848813',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:34:18Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='a327d22640aa4a7caeb35fb0ed2dccc8',ramdisk_id='',reservation_id='r-autn96ne',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-1144278074',owner_user_name='tempest-TaggedAttachmentsTest-1144278074-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:34:19Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3a7c2b6c06a745de95fae59f8b02c06d',uuid=37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "1fcf28a7-3fd9-4283-a0df-be0377b96d42", "address": "fa:16:3e:d3:72:8f", "network": {"id": "a1000896-9015-4517-a0b5-bd056cd9aa89", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-1097172279", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "a327d22640aa4a7caeb35fb0ed2dccc8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1fcf28a7-3f", "ovs_interfaceid": "1fcf28a7-3fd9-4283-a0df-be0377b96d42", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Converting VIF {"id": "1fcf28a7-3fd9-4283-a0df-be0377b96d42", "address": "fa:16:3e:d3:72:8f", "network": {"id": "a1000896-9015-4517-a0b5-bd056cd9aa89", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-1097172279", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "a327d22640aa4a7caeb35fb0ed2dccc8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1fcf28a7-3f", "ovs_interfaceid": "1fcf28a7-3fd9-4283-a0df-be0377b96d42", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d3:72:8f,bridge_name='br-int',has_traffic_filtering=True,id=1fcf28a7-3fd9-4283-a0df-be0377b96d42,network=Network(a1000896-9015-4517-a0b5-bd056cd9aa89),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1fcf28a7-3f') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:d3:72:8f"/><model type="virtio"/><mtu size="1442"/><target dev="tap1fcf28a7-3f"/></interface> {{(pid=72081) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:d3:72:8f"/><model type="virtio"/><mtu size="1442"/><target dev="tap1fcf28a7-3f"/></interface> {{(pid=72081) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Attempting to detach device tap1fcf28a7-3f from instance 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] detach device xml: <interface type="ethernet">
Apr 12 03:36:13 user nova-compute[72081]:   <mac address="fa:16:3e:d3:72:8f"/>
Apr 12 03:36:13 user nova-compute[72081]:   <model type="virtio"/>
Apr 12 03:36:13 user nova-compute[72081]:   <mtu size="1442"/>
Apr 12 03:36:13 user nova-compute[72081]:   <target dev="tap1fcf28a7-3f"/>
Apr 12 03:36:13 user nova-compute[72081]: </interface>
Apr 12 03:36:13 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:d3:72:8f"/><model type="virtio"/><mtu size="1442"/><target dev="tap1fcf28a7-3f"/></interface> {{(pid=72081) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] interface for config: <interface type="ethernet"><mac address="fa:16:3e:d3:72:8f"/><model type="virtio"/><mtu size="1442"/><target dev="tap1fcf28a7-3f"/></interface>not found in domain: <domain type='kvm' id='15'>
Apr 12 03:36:13 user nova-compute[72081]:   <name>instance-0000000f</name>
Apr 12 03:36:13 user nova-compute[72081]:   <uuid>37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63</uuid>
Apr 12 03:36:13 user nova-compute[72081]:   <metadata>
Apr 12 03:36:13 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1" xmlns:instance="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:36:13 user nova-compute[72081]:   <nova:package version="0.0.0"/>
Apr 12 03:36:13 user nova-compute[72081]:   <nova:name>tempest-device-tagging-server-934756275</nova:name>
Apr 12 03:36:13 user nova-compute[72081]:   <nova:creationTime>2023-04-12 03:36:01</nova:creationTime>
Apr 12 03:36:13 user nova-compute[72081]:   <nova:flavor name="m1.nano">
Apr 12 03:36:13 user nova-compute[72081]:     <nova:memory>128</nova:memory>
Apr 12 03:36:13 user nova-compute[72081]:     <nova:disk>1</nova:disk>
Apr 12 03:36:13 user nova-compute[72081]:     <nova:swap>0</nova:swap>
Apr 12 03:36:13 user nova-compute[72081]:     <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:36:13 user nova-compute[72081]:     <nova:vcpus>1</nova:vcpus>
Apr 12 03:36:13 user nova-compute[72081]:   </nova:flavor>
Apr 12 03:36:13 user nova-compute[72081]:   <nova:owner>
Apr 12 03:36:13 user nova-compute[72081]:     <nova:user uuid="3a7c2b6c06a745de95fae59f8b02c06d">tempest-TaggedAttachmentsTest-1144278074-project-member</nova:user>
Apr 12 03:36:13 user nova-compute[72081]:     <nova:project uuid="a327d22640aa4a7caeb35fb0ed2dccc8">tempest-TaggedAttachmentsTest-1144278074</nova:project>
Apr 12 03:36:13 user nova-compute[72081]:   </nova:owner>
Apr 12 03:36:13 user nova-compute[72081]:   <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:36:13 user nova-compute[72081]:   <nova:ports>
Apr 12 03:36:13 user nova-compute[72081]:     <nova:port uuid="b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958">
Apr 12 03:36:13 user nova-compute[72081]:       <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Apr 12 03:36:13 user nova-compute[72081]:     </nova:port>
Apr 12 03:36:13 user nova-compute[72081]:     <nova:port uuid="1fcf28a7-3fd9-4283-a0df-be0377b96d42">
Apr 12 03:36:13 user nova-compute[72081]:       <nova:ip type="fixed" address="10.10.10.34" ipVersion="4"/>
Apr 12 03:36:13 user nova-compute[72081]:     </nova:port>
Apr 12 03:36:13 user nova-compute[72081]:   </nova:ports>
Apr 12 03:36:13 user nova-compute[72081]: </nova:instance>
Apr 12 03:36:13 user nova-compute[72081]:   </metadata>
Apr 12 03:36:13 user nova-compute[72081]:   <memory unit='KiB'>131072</memory>
Apr 12 03:36:13 user nova-compute[72081]:   <currentMemory unit='KiB'>131072</currentMemory>
Apr 12 03:36:13 user nova-compute[72081]:   <vcpu placement='static'>1</vcpu>
Apr 12 03:36:13 user nova-compute[72081]:   <resource>
Apr 12 03:36:13 user nova-compute[72081]:     <partition>/machine</partition>
Apr 12 03:36:13 user nova-compute[72081]:   </resource>
Apr 12 03:36:13 user nova-compute[72081]:   <sysinfo type='smbios'>
Apr 12 03:36:13 user nova-compute[72081]:     <system>
Apr 12 03:36:13 user nova-compute[72081]:       <entry name='manufacturer'>OpenStack Foundation</entry>
Apr 12 03:36:13 user nova-compute[72081]:       <entry name='product'>OpenStack Nova</entry>
Apr 12 03:36:13 user nova-compute[72081]:       <entry name='version'>0.0.0</entry>
Apr 12 03:36:13 user nova-compute[72081]:       <entry name='serial'>37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63</entry>
Apr 12 03:36:13 user nova-compute[72081]:       <entry name='uuid'>37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63</entry>
Apr 12 03:36:13 user nova-compute[72081]:       <entry name='family'>Virtual Machine</entry>
Apr 12 03:36:13 user nova-compute[72081]:     </system>
Apr 12 03:36:13 user nova-compute[72081]:   </sysinfo>
Apr 12 03:36:13 user nova-compute[72081]:   <os>
Apr 12 03:36:13 user nova-compute[72081]:     <type arch='x86_64' machine='pc-i440fx-6.2'>hvm</type>
Apr 12 03:36:13 user nova-compute[72081]:     <boot dev='hd'/>
Apr 12 03:36:13 user nova-compute[72081]:     <smbios mode='sysinfo'/>
Apr 12 03:36:13 user nova-compute[72081]:   </os>
Apr 12 03:36:13 user nova-compute[72081]:   <features>
Apr 12 03:36:13 user nova-compute[72081]:     <acpi/>
Apr 12 03:36:13 user nova-compute[72081]:     <apic/>
Apr 12 03:36:13 user nova-compute[72081]:     <vmcoreinfo state='on'/>
Apr 12 03:36:13 user nova-compute[72081]:   </features>
Apr 12 03:36:13 user nova-compute[72081]:   <cpu mode='custom' match='exact' check='full'>
Apr 12 03:36:13 user nova-compute[72081]:     <model fallback='forbid'>Nehalem</model>
Apr 12 03:36:13 user nova-compute[72081]:     <topology sockets='1' dies='1' cores='1' threads='1'/>
Apr 12 03:36:13 user nova-compute[72081]:     <feature policy='require' name='vme'/>
Apr 12 03:36:13 user nova-compute[72081]:     <feature policy='require' name='x2apic'/>
Apr 12 03:36:13 user nova-compute[72081]:     <feature policy='require' name='hypervisor'/>
Apr 12 03:36:13 user nova-compute[72081]:   </cpu>
Apr 12 03:36:13 user nova-compute[72081]:   <clock offset='utc'>
Apr 12 03:36:13 user nova-compute[72081]:     <timer name='pit' tickpolicy='delay'/>
Apr 12 03:36:13 user nova-compute[72081]:     <timer name='rtc' tickpolicy='catchup'/>
Apr 12 03:36:13 user nova-compute[72081]:     <timer name='hpet' present='no'/>
Apr 12 03:36:13 user nova-compute[72081]:   </clock>
Apr 12 03:36:13 user nova-compute[72081]:   <on_poweroff>destroy</on_poweroff>
Apr 12 03:36:13 user nova-compute[72081]:   <on_reboot>restart</on_reboot>
Apr 12 03:36:13 user nova-compute[72081]:   <on_crash>destroy</on_crash>
Apr 12 03:36:13 user nova-compute[72081]:   <devices>
Apr 12 03:36:13 user nova-compute[72081]:     <emulator>/usr/bin/qemu-system-x86_64</emulator>
Apr 12 03:36:13 user nova-compute[72081]:     <disk type='file' device='disk'>
Apr 12 03:36:13 user nova-compute[72081]:       <driver name='qemu' type='qcow2' cache='none'/>
Apr 12 03:36:13 user nova-compute[72081]:       <source file='/opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk' index='2'/>
Apr 12 03:36:13 user nova-compute[72081]:       <backingStore type='file' index='3'>
Apr 12 03:36:13 user nova-compute[72081]:         <format type='raw'/>
Apr 12 03:36:13 user nova-compute[72081]:         <source file='/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df'/>
Apr 12 03:36:13 user nova-compute[72081]:         <backingStore/>
Apr 12 03:36:13 user nova-compute[72081]:       </backingStore>
Apr 12 03:36:13 user nova-compute[72081]:       <target dev='vda' bus='virtio'/>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='virtio-disk0'/>
Apr 12 03:36:13 user nova-compute[72081]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
Apr 12 03:36:13 user nova-compute[72081]:     </disk>
Apr 12 03:36:13 user nova-compute[72081]:     <disk type='file' device='cdrom'>
Apr 12 03:36:13 user nova-compute[72081]:       <driver name='qemu' type='raw' cache='none'/>
Apr 12 03:36:13 user nova-compute[72081]:       <source file='/opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk.config' index='1'/>
Apr 12 03:36:13 user nova-compute[72081]:       <backingStore/>
Apr 12 03:36:13 user nova-compute[72081]:       <target dev='hda' bus='ide'/>
Apr 12 03:36:13 user nova-compute[72081]:       <readonly/>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='ide0-0-0'/>
Apr 12 03:36:13 user nova-compute[72081]:       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
Apr 12 03:36:13 user nova-compute[72081]:     </disk>
Apr 12 03:36:13 user nova-compute[72081]:     <controller type='usb' index='0' model='none'>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='usb'/>
Apr 12 03:36:13 user nova-compute[72081]:     </controller>
Apr 12 03:36:13 user nova-compute[72081]:     <controller type='pci' index='0' model='pci-root'>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='pci.0'/>
Apr 12 03:36:13 user nova-compute[72081]:     </controller>
Apr 12 03:36:13 user nova-compute[72081]:     <controller type='ide' index='0'>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='ide'/>
Apr 12 03:36:13 user nova-compute[72081]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
Apr 12 03:36:13 user nova-compute[72081]:     </controller>
Apr 12 03:36:13 user nova-compute[72081]:     <interface type='ethernet'>
Apr 12 03:36:13 user nova-compute[72081]:       <mac address='fa:16:3e:46:28:01'/>
Apr 12 03:36:13 user nova-compute[72081]:       <target dev='tapb0b22dc0-c2'/>
Apr 12 03:36:13 user nova-compute[72081]:       <model type='virtio'/>
Apr 12 03:36:13 user nova-compute[72081]:       <mtu size='1442'/>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='net0'/>
Apr 12 03:36:13 user nova-compute[72081]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
Apr 12 03:36:13 user nova-compute[72081]:     </interface>
Apr 12 03:36:13 user nova-compute[72081]:     <interface type='ethernet'>
Apr 12 03:36:13 user nova-compute[72081]:       <mac address='fa:16:3e:d3:72:8f'/>
Apr 12 03:36:13 user nova-compute[72081]:       <target dev='tap1fcf28a7-3f'/>
Apr 12 03:36:13 user nova-compute[72081]:       <model type='virtio'/>
Apr 12 03:36:13 user nova-compute[72081]:       <mtu size='1442'/>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='net1'/>
Apr 12 03:36:13 user nova-compute[72081]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
Apr 12 03:36:13 user nova-compute[72081]:     </interface>
Apr 12 03:36:13 user nova-compute[72081]:     <serial type='pty'>
Apr 12 03:36:13 user nova-compute[72081]:       <source path='/dev/pts/4'/>
Apr 12 03:36:13 user nova-compute[72081]:       <log file='/opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/console.log' append='off'/>
Apr 12 03:36:13 user nova-compute[72081]:       <target type='isa-serial' port='0'>
Apr 12 03:36:13 user nova-compute[72081]:         <model name='isa-serial'/>
Apr 12 03:36:13 user nova-compute[72081]:       </target>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='serial0'/>
Apr 12 03:36:13 user nova-compute[72081]:     </serial>
Apr 12 03:36:13 user nova-compute[72081]:     <console type='pty' tty='/dev/pts/4'>
Apr 12 03:36:13 user nova-compute[72081]:       <source path='/dev/pts/4'/>
Apr 12 03:36:13 user nova-compute[72081]:       <log file='/opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/console.log' append='off'/>
Apr 12 03:36:13 user nova-compute[72081]:       <target type='serial' port='0'/>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='serial0'/>
Apr 12 03:36:13 user nova-compute[72081]:     </console>
Apr 12 03:36:13 user nova-compute[72081]:     <input type='mouse' bus='ps2'>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='input0'/>
Apr 12 03:36:13 user nova-compute[72081]:     </input>
Apr 12 03:36:13 user nova-compute[72081]:     <input type='keyboard' bus='ps2'>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='input1'/>
Apr 12 03:36:13 user nova-compute[72081]:     </input>
Apr 12 03:36:13 user nova-compute[72081]:     <graphics type='vnc' port='5905' autoport='yes' listen='0.0.0.0'>
Apr 12 03:36:13 user nova-compute[72081]:       <listen type='address' address='0.0.0.0'/>
Apr 12 03:36:13 user nova-compute[72081]:     </graphics>
Apr 12 03:36:13 user nova-compute[72081]:     <audio id='1' type='none'/>
Apr 12 03:36:13 user nova-compute[72081]:     <video>
Apr 12 03:36:13 user nova-compute[72081]:       <model type='virtio' heads='1' primary='yes'/>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='video0'/>
Apr 12 03:36:13 user nova-compute[72081]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
Apr 12 03:36:13 user nova-compute[72081]:     </video>
Apr 12 03:36:13 user nova-compute[72081]:     <memballoon model='virtio'>
Apr 12 03:36:13 user nova-compute[72081]:       <stats period='10'/>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='balloon0'/>
Apr 12 03:36:13 user nova-compute[72081]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
Apr 12 03:36:13 user nova-compute[72081]:     </memballoon>
Apr 12 03:36:13 user nova-compute[72081]:     <rng model='virtio'>
Apr 12 03:36:13 user nova-compute[72081]:       <backend model='random'>/dev/urandom</backend>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='rng0'/>
Apr 12 03:36:13 user nova-compute[72081]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
Apr 12 03:36:13 user nova-compute[72081]:     </rng>
Apr 12 03:36:13 user nova-compute[72081]:   </devices>
Apr 12 03:36:13 user nova-compute[72081]:   <seclabel type='dynamic' model='apparmor' relabel='yes'>
Apr 12 03:36:13 user nova-compute[72081]:     <label>libvirt-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63</label>
Apr 12 03:36:13 user nova-compute[72081]:     <imagelabel>libvirt-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63</imagelabel>
Apr 12 03:36:13 user nova-compute[72081]:   </seclabel>
Apr 12 03:36:13 user nova-compute[72081]:   <seclabel type='dynamic' model='dac' relabel='yes'>
Apr 12 03:36:13 user nova-compute[72081]:     <label>+64055:+108</label>
Apr 12 03:36:13 user nova-compute[72081]:     <imagelabel>+64055:+108</imagelabel>
Apr 12 03:36:13 user nova-compute[72081]:   </seclabel>
Apr 12 03:36:13 user nova-compute[72081]: </domain>
Apr 12 03:36:13 user nova-compute[72081]:  {{(pid=72081) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:282}}
Apr 12 03:36:13 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Successfully detached device tap1fcf28a7-3f from instance 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 from the persistent domain config.
Apr 12 03:36:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] (1/8): Attempting to detach device tap1fcf28a7-3f with device alias net1 from instance 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 from the live domain config. {{(pid=72081) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] detach device xml: <interface type="ethernet">
Apr 12 03:36:13 user nova-compute[72081]:   <mac address="fa:16:3e:d3:72:8f"/>
Apr 12 03:36:13 user nova-compute[72081]:   <model type="virtio"/>
Apr 12 03:36:13 user nova-compute[72081]:   <mtu size="1442"/>
Apr 12 03:36:13 user nova-compute[72081]:   <target dev="tap1fcf28a7-3f"/>
Apr 12 03:36:13 user nova-compute[72081]: </interface>
Apr 12 03:36:13 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Received event <DeviceRemovedEvent: 1681270573.6430645, 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 => net1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72081) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Start waiting for the detach event from libvirt for device tap1fcf28a7-3f with device alias net1 for instance 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 {{(pid=72081) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:d3:72:8f"/><model type="virtio"/><mtu size="1442"/><target dev="tap1fcf28a7-3f"/></interface> {{(pid=72081) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] interface for config: <interface type="ethernet"><mac address="fa:16:3e:d3:72:8f"/><model type="virtio"/><mtu size="1442"/><target dev="tap1fcf28a7-3f"/></interface>not found in domain: <domain type='kvm' id='15'>
Apr 12 03:36:13 user nova-compute[72081]:   <name>instance-0000000f</name>
Apr 12 03:36:13 user nova-compute[72081]:   <uuid>37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63</uuid>
Apr 12 03:36:13 user nova-compute[72081]:   <metadata>
Apr 12 03:36:13 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1" xmlns:instance="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:36:13 user nova-compute[72081]:   <nova:package version="0.0.0"/>
Apr 12 03:36:13 user nova-compute[72081]:   <nova:name>tempest-device-tagging-server-934756275</nova:name>
Apr 12 03:36:13 user nova-compute[72081]:   <nova:creationTime>2023-04-12 03:36:01</nova:creationTime>
Apr 12 03:36:13 user nova-compute[72081]:   <nova:flavor name="m1.nano">
Apr 12 03:36:13 user nova-compute[72081]:     <nova:memory>128</nova:memory>
Apr 12 03:36:13 user nova-compute[72081]:     <nova:disk>1</nova:disk>
Apr 12 03:36:13 user nova-compute[72081]:     <nova:swap>0</nova:swap>
Apr 12 03:36:13 user nova-compute[72081]:     <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:36:13 user nova-compute[72081]:     <nova:vcpus>1</nova:vcpus>
Apr 12 03:36:13 user nova-compute[72081]:   </nova:flavor>
Apr 12 03:36:13 user nova-compute[72081]:   <nova:owner>
Apr 12 03:36:13 user nova-compute[72081]:     <nova:user uuid="3a7c2b6c06a745de95fae59f8b02c06d">tempest-TaggedAttachmentsTest-1144278074-project-member</nova:user>
Apr 12 03:36:13 user nova-compute[72081]:     <nova:project uuid="a327d22640aa4a7caeb35fb0ed2dccc8">tempest-TaggedAttachmentsTest-1144278074</nova:project>
Apr 12 03:36:13 user nova-compute[72081]:   </nova:owner>
Apr 12 03:36:13 user nova-compute[72081]:   <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:36:13 user nova-compute[72081]:   <nova:ports>
Apr 12 03:36:13 user nova-compute[72081]:     <nova:port uuid="b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958">
Apr 12 03:36:13 user nova-compute[72081]:       <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Apr 12 03:36:13 user nova-compute[72081]:     </nova:port>
Apr 12 03:36:13 user nova-compute[72081]:     <nova:port uuid="1fcf28a7-3fd9-4283-a0df-be0377b96d42">
Apr 12 03:36:13 user nova-compute[72081]:       <nova:ip type="fixed" address="10.10.10.34" ipVersion="4"/>
Apr 12 03:36:13 user nova-compute[72081]:     </nova:port>
Apr 12 03:36:13 user nova-compute[72081]:   </nova:ports>
Apr 12 03:36:13 user nova-compute[72081]: </nova:instance>
Apr 12 03:36:13 user nova-compute[72081]:   </metadata>
Apr 12 03:36:13 user nova-compute[72081]:   <memory unit='KiB'>131072</memory>
Apr 12 03:36:13 user nova-compute[72081]:   <currentMemory unit='KiB'>131072</currentMemory>
Apr 12 03:36:13 user nova-compute[72081]:   <vcpu placement='static'>1</vcpu>
Apr 12 03:36:13 user nova-compute[72081]:   <resource>
Apr 12 03:36:13 user nova-compute[72081]:     <partition>/machine</partition>
Apr 12 03:36:13 user nova-compute[72081]:   </resource>
Apr 12 03:36:13 user nova-compute[72081]:   <sysinfo type='smbios'>
Apr 12 03:36:13 user nova-compute[72081]:     <system>
Apr 12 03:36:13 user nova-compute[72081]:       <entry name='manufacturer'>OpenStack Foundation</entry>
Apr 12 03:36:13 user nova-compute[72081]:       <entry name='product'>OpenStack Nova</entry>
Apr 12 03:36:13 user nova-compute[72081]:       <entry name='version'>0.0.0</entry>
Apr 12 03:36:13 user nova-compute[72081]:       <entry name='serial'>37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63</entry>
Apr 12 03:36:13 user nova-compute[72081]:       <entry name='uuid'>37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63</entry>
Apr 12 03:36:13 user nova-compute[72081]:       <entry name='family'>Virtual Machine</entry>
Apr 12 03:36:13 user nova-compute[72081]:     </system>
Apr 12 03:36:13 user nova-compute[72081]:   </sysinfo>
Apr 12 03:36:13 user nova-compute[72081]:   <os>
Apr 12 03:36:13 user nova-compute[72081]:     <type arch='x86_64' machine='pc-i440fx-6.2'>hvm</type>
Apr 12 03:36:13 user nova-compute[72081]:     <boot dev='hd'/>
Apr 12 03:36:13 user nova-compute[72081]:     <smbios mode='sysinfo'/>
Apr 12 03:36:13 user nova-compute[72081]:   </os>
Apr 12 03:36:13 user nova-compute[72081]:   <features>
Apr 12 03:36:13 user nova-compute[72081]:     <acpi/>
Apr 12 03:36:13 user nova-compute[72081]:     <apic/>
Apr 12 03:36:13 user nova-compute[72081]:     <vmcoreinfo state='on'/>
Apr 12 03:36:13 user nova-compute[72081]:   </features>
Apr 12 03:36:13 user nova-compute[72081]:   <cpu mode='custom' match='exact' check='full'>
Apr 12 03:36:13 user nova-compute[72081]:     <model fallback='forbid'>Nehalem</model>
Apr 12 03:36:13 user nova-compute[72081]:     <topology sockets='1' dies='1' cores='1' threads='1'/>
Apr 12 03:36:13 user nova-compute[72081]:     <feature policy='require' name='vme'/>
Apr 12 03:36:13 user nova-compute[72081]:     <feature policy='require' name='x2apic'/>
Apr 12 03:36:13 user nova-compute[72081]:     <feature policy='require' name='hypervisor'/>
Apr 12 03:36:13 user nova-compute[72081]:   </cpu>
Apr 12 03:36:13 user nova-compute[72081]:   <clock offset='utc'>
Apr 12 03:36:13 user nova-compute[72081]:     <timer name='pit' tickpolicy='delay'/>
Apr 12 03:36:13 user nova-compute[72081]:     <timer name='rtc' tickpolicy='catchup'/>
Apr 12 03:36:13 user nova-compute[72081]:     <timer name='hpet' present='no'/>
Apr 12 03:36:13 user nova-compute[72081]:   </clock>
Apr 12 03:36:13 user nova-compute[72081]:   <on_poweroff>destroy</on_poweroff>
Apr 12 03:36:13 user nova-compute[72081]:   <on_reboot>restart</on_reboot>
Apr 12 03:36:13 user nova-compute[72081]:   <on_crash>destroy</on_crash>
Apr 12 03:36:13 user nova-compute[72081]:   <devices>
Apr 12 03:36:13 user nova-compute[72081]:     <emulator>/usr/bin/qemu-system-x86_64</emulator>
Apr 12 03:36:13 user nova-compute[72081]:     <disk type='file' device='disk'>
Apr 12 03:36:13 user nova-compute[72081]:       <driver name='qemu' type='qcow2' cache='none'/>
Apr 12 03:36:13 user nova-compute[72081]:       <source file='/opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk' index='2'/>
Apr 12 03:36:13 user nova-compute[72081]:       <backingStore type='file' index='3'>
Apr 12 03:36:13 user nova-compute[72081]:         <format type='raw'/>
Apr 12 03:36:13 user nova-compute[72081]:         <source file='/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df'/>
Apr 12 03:36:13 user nova-compute[72081]:         <backingStore/>
Apr 12 03:36:13 user nova-compute[72081]:       </backingStore>
Apr 12 03:36:13 user nova-compute[72081]:       <target dev='vda' bus='virtio'/>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='virtio-disk0'/>
Apr 12 03:36:13 user nova-compute[72081]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
Apr 12 03:36:13 user nova-compute[72081]:     </disk>
Apr 12 03:36:13 user nova-compute[72081]:     <disk type='file' device='cdrom'>
Apr 12 03:36:13 user nova-compute[72081]:       <driver name='qemu' type='raw' cache='none'/>
Apr 12 03:36:13 user nova-compute[72081]:       <source file='/opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/disk.config' index='1'/>
Apr 12 03:36:13 user nova-compute[72081]:       <backingStore/>
Apr 12 03:36:13 user nova-compute[72081]:       <target dev='hda' bus='ide'/>
Apr 12 03:36:13 user nova-compute[72081]:       <readonly/>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='ide0-0-0'/>
Apr 12 03:36:13 user nova-compute[72081]:       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
Apr 12 03:36:13 user nova-compute[72081]:     </disk>
Apr 12 03:36:13 user nova-compute[72081]:     <controller type='usb' index='0' model='none'>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='usb'/>
Apr 12 03:36:13 user nova-compute[72081]:     </controller>
Apr 12 03:36:13 user nova-compute[72081]:     <controller type='pci' index='0' model='pci-root'>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='pci.0'/>
Apr 12 03:36:13 user nova-compute[72081]:     </controller>
Apr 12 03:36:13 user nova-compute[72081]:     <controller type='ide' index='0'>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='ide'/>
Apr 12 03:36:13 user nova-compute[72081]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
Apr 12 03:36:13 user nova-compute[72081]:     </controller>
Apr 12 03:36:13 user nova-compute[72081]:     <interface type='ethernet'>
Apr 12 03:36:13 user nova-compute[72081]:       <mac address='fa:16:3e:46:28:01'/>
Apr 12 03:36:13 user nova-compute[72081]:       <target dev='tapb0b22dc0-c2'/>
Apr 12 03:36:13 user nova-compute[72081]:       <model type='virtio'/>
Apr 12 03:36:13 user nova-compute[72081]:       <mtu size='1442'/>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='net0'/>
Apr 12 03:36:13 user nova-compute[72081]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
Apr 12 03:36:13 user nova-compute[72081]:     </interface>
Apr 12 03:36:13 user nova-compute[72081]:     <serial type='pty'>
Apr 12 03:36:13 user nova-compute[72081]:       <source path='/dev/pts/4'/>
Apr 12 03:36:13 user nova-compute[72081]:       <log file='/opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/console.log' append='off'/>
Apr 12 03:36:13 user nova-compute[72081]:       <target type='isa-serial' port='0'>
Apr 12 03:36:13 user nova-compute[72081]:         <model name='isa-serial'/>
Apr 12 03:36:13 user nova-compute[72081]:       </target>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='serial0'/>
Apr 12 03:36:13 user nova-compute[72081]:     </serial>
Apr 12 03:36:13 user nova-compute[72081]:     <console type='pty' tty='/dev/pts/4'>
Apr 12 03:36:13 user nova-compute[72081]:       <source path='/dev/pts/4'/>
Apr 12 03:36:13 user nova-compute[72081]:       <log file='/opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63/console.log' append='off'/>
Apr 12 03:36:13 user nova-compute[72081]:       <target type='serial' port='0'/>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='serial0'/>
Apr 12 03:36:13 user nova-compute[72081]:     </console>
Apr 12 03:36:13 user nova-compute[72081]:     <input type='mouse' bus='ps2'>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='input0'/>
Apr 12 03:36:13 user nova-compute[72081]:     </input>
Apr 12 03:36:13 user nova-compute[72081]:     <input type='keyboard' bus='ps2'>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='input1'/>
Apr 12 03:36:13 user nova-compute[72081]:     </input>
Apr 12 03:36:13 user nova-compute[72081]:     <graphics type='vnc' port='5905' autoport='yes' listen='0.0.0.0'>
Apr 12 03:36:13 user nova-compute[72081]:       <listen type='address' address='0.0.0.0'/>
Apr 12 03:36:13 user nova-compute[72081]:     </graphics>
Apr 12 03:36:13 user nova-compute[72081]:     <audio id='1' type='none'/>
Apr 12 03:36:13 user nova-compute[72081]:     <video>
Apr 12 03:36:13 user nova-compute[72081]:       <model type='virtio' heads='1' primary='yes'/>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='video0'/>
Apr 12 03:36:13 user nova-compute[72081]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
Apr 12 03:36:13 user nova-compute[72081]:     </video>
Apr 12 03:36:13 user nova-compute[72081]:     <memballoon model='virtio'>
Apr 12 03:36:13 user nova-compute[72081]:       <stats period='10'/>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='balloon0'/>
Apr 12 03:36:13 user nova-compute[72081]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
Apr 12 03:36:13 user nova-compute[72081]:     </memballoon>
Apr 12 03:36:13 user nova-compute[72081]:     <rng model='virtio'>
Apr 12 03:36:13 user nova-compute[72081]:       <backend model='random'>/dev/urandom</backend>
Apr 12 03:36:13 user nova-compute[72081]:       <alias name='rng0'/>
Apr 12 03:36:13 user nova-compute[72081]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
Apr 12 03:36:13 user nova-compute[72081]:     </rng>
Apr 12 03:36:13 user nova-compute[72081]:   </devices>
Apr 12 03:36:13 user nova-compute[72081]:   <seclabel type='dynamic' model='apparmor' relabel='yes'>
Apr 12 03:36:13 user nova-compute[72081]:     <label>libvirt-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63</label>
Apr 12 03:36:13 user nova-compute[72081]:     <imagelabel>libvirt-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63</imagelabel>
Apr 12 03:36:13 user nova-compute[72081]:   </seclabel>
Apr 12 03:36:13 user nova-compute[72081]:   <seclabel type='dynamic' model='dac' relabel='yes'>
Apr 12 03:36:13 user nova-compute[72081]:     <label>+64055:+108</label>
Apr 12 03:36:13 user nova-compute[72081]:     <imagelabel>+64055:+108</imagelabel>
Apr 12 03:36:13 user nova-compute[72081]:   </seclabel>
Apr 12 03:36:13 user nova-compute[72081]: </domain>
Apr 12 03:36:13 user nova-compute[72081]:  {{(pid=72081) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:282}}
Apr 12 03:36:13 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Successfully detached device tap1fcf28a7-3f from instance 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 from the live domain config.
Apr 12 03:36:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:34:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=InstanceDeviceMetadata,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-934756275',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-934756275',id=15,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGqZ2JOd0oMveikCRo3egpYXw6F7M9CSnvg8uuSzQFHFmYKOQ1+3ysT+GbPl8KtibugwjCH/8obQl0aNd57dyywpuoimEfD8++mntdrmImk7chBwlJ4Kqnvt9BRQOwzT8Q==',key_name='tempest-keypair-52848813',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:34:18Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='a327d22640aa4a7caeb35fb0ed2dccc8',ramdisk_id='',reservation_id='r-autn96ne',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-1144278074',owner_user_name='tempest-TaggedAttachmentsTest-1144278074-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:34:19Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3a7c2b6c06a745de95fae59f8b02c06d',uuid=37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "1fcf28a7-3fd9-4283-a0df-be0377b96d42", "address": "fa:16:3e:d3:72:8f", "network": {"id": "a1000896-9015-4517-a0b5-bd056cd9aa89", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-1097172279", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "a327d22640aa4a7caeb35fb0ed2dccc8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1fcf28a7-3f", "ovs_interfaceid": "1fcf28a7-3fd9-4283-a0df-be0377b96d42", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Converting VIF {"id": "1fcf28a7-3fd9-4283-a0df-be0377b96d42", "address": "fa:16:3e:d3:72:8f", "network": {"id": "a1000896-9015-4517-a0b5-bd056cd9aa89", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-1097172279", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "a327d22640aa4a7caeb35fb0ed2dccc8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1fcf28a7-3f", "ovs_interfaceid": "1fcf28a7-3fd9-4283-a0df-be0377b96d42", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d3:72:8f,bridge_name='br-int',has_traffic_filtering=True,id=1fcf28a7-3fd9-4283-a0df-be0377b96d42,network=Network(a1000896-9015-4517-a0b5-bd056cd9aa89),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1fcf28a7-3f') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG os_vif [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:d3:72:8f,bridge_name='br-int',has_traffic_filtering=True,id=1fcf28a7-3fd9-4283-a0df-be0377b96d42,network=Network(a1000896-9015-4517-a0b5-bd056cd9aa89),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1fcf28a7-3f') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap1fcf28a7-3f, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:36:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:13 user nova-compute[72081]: INFO os_vif [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:d3:72:8f,bridge_name='br-int',has_traffic_filtering=True,id=1fcf28a7-3fd9-4283-a0df-be0377b96d42,network=Network(a1000896-9015-4517-a0b5-bd056cd9aa89),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1fcf28a7-3f')
Apr 12 03:36:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] set metadata xml: <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:36:13 user nova-compute[72081]:   <nova:package version="0.0.0"/>
Apr 12 03:36:13 user nova-compute[72081]:   <nova:name>tempest-device-tagging-server-934756275</nova:name>
Apr 12 03:36:13 user nova-compute[72081]:   <nova:creationTime>2023-04-12 03:36:13</nova:creationTime>
Apr 12 03:36:13 user nova-compute[72081]:   <nova:flavor name="m1.nano">
Apr 12 03:36:13 user nova-compute[72081]:     <nova:memory>128</nova:memory>
Apr 12 03:36:13 user nova-compute[72081]:     <nova:disk>1</nova:disk>
Apr 12 03:36:13 user nova-compute[72081]:     <nova:swap>0</nova:swap>
Apr 12 03:36:13 user nova-compute[72081]:     <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:36:13 user nova-compute[72081]:     <nova:vcpus>1</nova:vcpus>
Apr 12 03:36:13 user nova-compute[72081]:   </nova:flavor>
Apr 12 03:36:13 user nova-compute[72081]:   <nova:owner>
Apr 12 03:36:13 user nova-compute[72081]:     <nova:user uuid="3a7c2b6c06a745de95fae59f8b02c06d">tempest-TaggedAttachmentsTest-1144278074-project-member</nova:user>
Apr 12 03:36:13 user nova-compute[72081]:     <nova:project uuid="a327d22640aa4a7caeb35fb0ed2dccc8">tempest-TaggedAttachmentsTest-1144278074</nova:project>
Apr 12 03:36:13 user nova-compute[72081]:   </nova:owner>
Apr 12 03:36:13 user nova-compute[72081]:   <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:36:13 user nova-compute[72081]:   <nova:ports>
Apr 12 03:36:13 user nova-compute[72081]:     <nova:port uuid="b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958">
Apr 12 03:36:13 user nova-compute[72081]:       <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Apr 12 03:36:13 user nova-compute[72081]:     </nova:port>
Apr 12 03:36:13 user nova-compute[72081]:   </nova:ports>
Apr 12 03:36:13 user nova-compute[72081]: </nova:instance>
Apr 12 03:36:13 user nova-compute[72081]:  {{(pid=72081) set_metadata /opt/stack/nova/nova/virt/libvirt/guest.py:359}}
Apr 12 03:36:14 user nova-compute[72081]: DEBUG nova.compute.manager [req-df096b6e-891a-49ac-8cda-e0aa94df0d35 req-9053cdd5-c0c6-40d2-b998-f9d5d7f59343 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Received event network-vif-unplugged-1fcf28a7-3fd9-4283-a0df-be0377b96d42 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-df096b6e-891a-49ac-8cda-e0aa94df0d35 req-9053cdd5-c0c6-40d2-b998-f9d5d7f59343 service nova] Acquiring lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-df096b6e-891a-49ac-8cda-e0aa94df0d35 req-9053cdd5-c0c6-40d2-b998-f9d5d7f59343 service nova] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-df096b6e-891a-49ac-8cda-e0aa94df0d35 req-9053cdd5-c0c6-40d2-b998-f9d5d7f59343 service nova] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:14 user nova-compute[72081]: DEBUG nova.compute.manager [req-df096b6e-891a-49ac-8cda-e0aa94df0d35 req-9053cdd5-c0c6-40d2-b998-f9d5d7f59343 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] No waiting events found dispatching network-vif-unplugged-1fcf28a7-3fd9-4283-a0df-be0377b96d42 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:14 user nova-compute[72081]: WARNING nova.compute.manager [req-df096b6e-891a-49ac-8cda-e0aa94df0d35 req-9053cdd5-c0c6-40d2-b998-f9d5d7f59343 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Received unexpected event network-vif-unplugged-1fcf28a7-3fd9-4283-a0df-be0377b96d42 for instance with vm_state active and task_state None.
Apr 12 03:36:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Acquiring lock "refresh_cache-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:36:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Acquired lock "refresh_cache-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:36:14 user nova-compute[72081]: DEBUG nova.network.neutron [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:36:15 user nova-compute[72081]: INFO nova.network.neutron [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Port 1fcf28a7-3fd9-4283-a0df-be0377b96d42 from network info_cache is no longer associated with instance in Neutron. Removing from network info_cache.
Apr 12 03:36:15 user nova-compute[72081]: DEBUG nova.network.neutron [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Updating instance_info_cache with network_info: [{"id": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "address": "fa:16:3e:46:28:01", "network": {"id": "8d284b3a-85a8-45a3-845f-271f3fe80540", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-354475413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.20", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a327d22640aa4a7caeb35fb0ed2dccc8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb0b22dc0-c2", "ovs_interfaceid": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:36:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Releasing lock "refresh_cache-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:36:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dba44164-f7f6-4550-b906-aa24f47058fc tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "interface-37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-1fcf28a7-3fd9-4283-a0df-be0377b96d42" "released" by "nova.compute.manager.ComputeManager.detach_interface.<locals>.do_detach_interface" :: held 2.020s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:16 user nova-compute[72081]: DEBUG nova.compute.manager [req-279ef1bb-028f-4fad-a6eb-43dbe15583c9 req-cf92f910-7b25-4726-9f77-1f7f94a57b52 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Received event network-vif-plugged-1fcf28a7-3fd9-4283-a0df-be0377b96d42 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-279ef1bb-028f-4fad-a6eb-43dbe15583c9 req-cf92f910-7b25-4726-9f77-1f7f94a57b52 service nova] Acquiring lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-279ef1bb-028f-4fad-a6eb-43dbe15583c9 req-cf92f910-7b25-4726-9f77-1f7f94a57b52 service nova] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-279ef1bb-028f-4fad-a6eb-43dbe15583c9 req-cf92f910-7b25-4726-9f77-1f7f94a57b52 service nova] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:16 user nova-compute[72081]: DEBUG nova.compute.manager [req-279ef1bb-028f-4fad-a6eb-43dbe15583c9 req-cf92f910-7b25-4726-9f77-1f7f94a57b52 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] No waiting events found dispatching network-vif-plugged-1fcf28a7-3fd9-4283-a0df-be0377b96d42 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:16 user nova-compute[72081]: WARNING nova.compute.manager [req-279ef1bb-028f-4fad-a6eb-43dbe15583c9 req-cf92f910-7b25-4726-9f77-1f7f94a57b52 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Received unexpected event network-vif-plugged-1fcf28a7-3fd9-4283-a0df-be0377b96d42 for instance with vm_state active and task_state None.
Apr 12 03:36:16 user nova-compute[72081]: DEBUG nova.compute.manager [req-279ef1bb-028f-4fad-a6eb-43dbe15583c9 req-cf92f910-7b25-4726-9f77-1f7f94a57b52 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Received event network-vif-deleted-1fcf28a7-3fd9-4283-a0df-be0377b96d42 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:19 user nova-compute[72081]: INFO nova.compute.manager [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Detaching volume b48ff4ab-9969-4337-bce5-6dcdfced462a
Apr 12 03:36:20 user nova-compute[72081]: INFO nova.virt.block_device [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Attempting to driver detach volume b48ff4ab-9969-4337-bce5-6dcdfced462a from mountpoint /dev/vdb
Apr 12 03:36:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Attempting to detach device vdb from instance 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:36:20 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:36:20 user nova-compute[72081]:   <source dev="/dev/sdf"/>
Apr 12 03:36:20 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:36:20 user nova-compute[72081]:   <serial>b48ff4ab-9969-4337-bce5-6dcdfced462a</serial>
Apr 12 03:36:20 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 03:36:20 user nova-compute[72081]: </disk>
Apr 12 03:36:20 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:36:20 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Successfully detached device vdb from instance 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f from the persistent domain config.
Apr 12 03:36:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f from the live domain config. {{(pid=72081) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:36:20 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:36:20 user nova-compute[72081]:   <source dev="/dev/sdf"/>
Apr 12 03:36:20 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:36:20 user nova-compute[72081]:   <serial>b48ff4ab-9969-4337-bce5-6dcdfced462a</serial>
Apr 12 03:36:20 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 03:36:20 user nova-compute[72081]: </disk>
Apr 12 03:36:20 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Received event <DeviceRemovedEvent: 1681270580.545668, 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72081) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f {{(pid=72081) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 12 03:36:20 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Successfully detached device vdb from instance 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f from the live domain config.
Apr 12 03:36:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ulsJFnqz', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:36:20 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:36:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[21b69bd6-9653-464c-a105-fe19178cf244]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:36:20 user nova-compute[72081]: No targets found.
Apr 12 03:36:20 user nova-compute[72081]: iSNS:
Apr 12 03:36:20 user nova-compute[72081]: No targets found.
Apr 12 03:36:20 user nova-compute[72081]: STATIC:
Apr 12 03:36:20 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a
Apr 12 03:36:20 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:36:20 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:36:20 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:36:20 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:36:20 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:36:20 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f
Apr 12 03:36:20 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:36:20 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:36:20 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939
Apr 12 03:36:20 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:36:20 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:36:20 user nova-compute[72081]: FIRMWARE:
Apr 12 03:36:20 user nova-compute[72081]: No targets found.
Apr 12 03:36:20 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:36:20 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:36:20 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d3299d3f-23f7-41c5-80a3-b641033272ed]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4786022e-c4eb-4800-a12e-4b352521b08e]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:20 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:36:20 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:20 user nova-compute[72081]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a (non-flash)
Apr 12 03:36:20 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:20 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:36:20 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:20 user nova-compute[72081]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a (non-flash)
Apr 12 03:36:20 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3f7473c310>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a'): ({'sdf'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Removing single pathed devices sdf {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[fc106e15-49f7-4364-8152-6c17cf7ed7e0]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdf {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdf" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[79bf3eda-9623-483e-b422-587658bf108d]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Flushing IO for device /dev/sdf {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdf {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdf" returned: 0 in 0.013s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d6d39fc5-b8d8-49ad-884a-a042e47b5b0d]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Remove SCSI device /dev/sdf with /sys/block/sdf/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdf/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdf/device/delete" returned: 0 in 0.049s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a6510e65-406d-453c-824c-1bf6dd6ffda7]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Checking to see if SCSI volumes sdf have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] SCSI volumes sdf have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9eab6d69-8079-4503-b557-21a4da6612eb]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a -p 172.16.0.220:3260 --logout" returned: 0 in 0.054s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7d3ff70b-eaa8-4984-8bc5-1633dc98548f]: (4, ('Logging out of session [sid: 19, target: iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a, portal: 172.16.0.220,3260]\nLogout of [sid: 19, target: iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 19, target: iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a, portal: 172.16.0.220,3260]
Apr 12 03:36:20 user nova-compute[72081]: Logout of [sid: 19, target: iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a, portal: 172.16.0.220,3260] successful.
Apr 12 03:36:20 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a -p 172.16.0.220:3260 --op delete" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[77017efe-321d-40aa-948a-dd3fde432809]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.305s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] <== disconnect_volume: return (306ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:36:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:36:21 user nova-compute[72081]: DEBUG nova.objects.instance [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lazy-loading 'flavor' on Instance uuid 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:36:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a47fc0a1-df50-4a54-b859-2da8068f3981 tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.874s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 1.742s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:21 user nova-compute[72081]: INFO nova.compute.manager [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Terminating instance
Apr 12 03:36:21 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:36:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG nova.compute.manager [req-561cb056-3ee3-458c-b57a-19d2d75edd3e req-18874d17-8c48-4913-90dc-4755c145acfb service nova] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Received event network-vif-unplugged-e3161de1-7643-4b1a-8a0c-27791fed8dbd {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-561cb056-3ee3-458c-b57a-19d2d75edd3e req-18874d17-8c48-4913-90dc-4755c145acfb service nova] Acquiring lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-561cb056-3ee3-458c-b57a-19d2d75edd3e req-18874d17-8c48-4913-90dc-4755c145acfb service nova] Lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-561cb056-3ee3-458c-b57a-19d2d75edd3e req-18874d17-8c48-4913-90dc-4755c145acfb service nova] Lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG nova.compute.manager [req-561cb056-3ee3-458c-b57a-19d2d75edd3e req-18874d17-8c48-4913-90dc-4755c145acfb service nova] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] No waiting events found dispatching network-vif-unplugged-e3161de1-7643-4b1a-8a0c-27791fed8dbd {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG nova.compute.manager [req-561cb056-3ee3-458c-b57a-19d2d75edd3e req-18874d17-8c48-4913-90dc-4755c145acfb service nova] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Received event network-vif-unplugged-e3161de1-7643-4b1a-8a0c-27791fed8dbd for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "23367b24-0a3b-454d-901d-8a3e5396333a" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "23367b24-0a3b-454d-901d-8a3e5396333a" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:22 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Instance destroyed successfully.
Apr 12 03:36:22 user nova-compute[72081]: DEBUG nova.objects.instance [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lazy-loading 'resources' on Instance uuid 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:34:12Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-99867029',display_name='tempest-VolumesSnapshotTestJSON-instance-99867029',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-99867029',id=16,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:34:23Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='86f7b42b161f41efb234f0ce70210a08',ramdisk_id='',reservation_id='r-r7onnebb',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesSnapshotTestJSON-1513599190',owner_user_name='tempest-VolumesSnapshotTestJSON-1513599190-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:34:23Z,user_data=None,user_id='84ea3a8776a944178ba5a04bce4d9b05',uuid=3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e3161de1-7643-4b1a-8a0c-27791fed8dbd", "address": "fa:16:3e:71:14:95", "network": {"id": "07d0f97b-06f9-40b3-8794-e53924379d25", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-2076424457-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "86f7b42b161f41efb234f0ce70210a08", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3161de1-76", "ovs_interfaceid": "e3161de1-7643-4b1a-8a0c-27791fed8dbd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Converting VIF {"id": "e3161de1-7643-4b1a-8a0c-27791fed8dbd", "address": "fa:16:3e:71:14:95", "network": {"id": "07d0f97b-06f9-40b3-8794-e53924379d25", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-2076424457-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "86f7b42b161f41efb234f0ce70210a08", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape3161de1-76", "ovs_interfaceid": "e3161de1-7643-4b1a-8a0c-27791fed8dbd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:71:14:95,bridge_name='br-int',has_traffic_filtering=True,id=e3161de1-7643-4b1a-8a0c-27791fed8dbd,network=Network(07d0f97b-06f9-40b3-8794-e53924379d25),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape3161de1-76') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG os_vif [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:71:14:95,bridge_name='br-int',has_traffic_filtering=True,id=e3161de1-7643-4b1a-8a0c-27791fed8dbd,network=Network(07d0f97b-06f9-40b3-8794-e53924379d25),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape3161de1-76') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tape3161de1-76, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lazy-loading 'flavor' on Instance uuid 23367b24-0a3b-454d-901d-8a3e5396333a {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:36:22 user nova-compute[72081]: INFO os_vif [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:71:14:95,bridge_name='br-int',has_traffic_filtering=True,id=e3161de1-7643-4b1a-8a0c-27791fed8dbd,network=Network(07d0f97b-06f9-40b3-8794-e53924379d25),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape3161de1-76')
Apr 12 03:36:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ulsJFnqz', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:36:22 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:36:22 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[459c44ce-224a-40f2-b58d-f3e7f7056323]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:36:22 user nova-compute[72081]: No targets found.
Apr 12 03:36:22 user nova-compute[72081]: iSNS:
Apr 12 03:36:22 user nova-compute[72081]: No targets found.
Apr 12 03:36:22 user nova-compute[72081]: STATIC:
Apr 12 03:36:22 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:36:22 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:36:22 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:36:22 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f
Apr 12 03:36:22 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:36:22 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:36:22 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939
Apr 12 03:36:22 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:36:22 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:36:22 user nova-compute[72081]: FIRMWARE:
Apr 12 03:36:22 user nova-compute[72081]: No targets found.
Apr 12 03:36:22 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:36:22 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:36:22 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b48ff4ab-9969-4337-bce5-6dcdfced462a', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "23367b24-0a3b-454d-901d-8a3e5396333a" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.087s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[354e685c-5b1d-4485-b46c-01a70f04602d]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ad87480d-971c-48b3-bd57-538cb61191fb]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:22 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:36:22 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:22 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:22 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:36:22 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:22 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fb6faa710>, {}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Disconnecting from: [] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.077s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] <== disconnect_volume: return (78ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:36:22 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Deleting instance files /opt/stack/data/nova/instances/3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f_del
Apr 12 03:36:22 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Deletion of /opt/stack/data/nova/instances/3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f_del complete
Apr 12 03:36:22 user nova-compute[72081]: INFO nova.compute.manager [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Took 0.95 seconds to destroy the instance on the hypervisor.
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "23367b24-0a3b-454d-901d-8a3e5396333a" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "23367b24-0a3b-454d-901d-8a3e5396333a" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:22 user nova-compute[72081]: INFO nova.compute.manager [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Attaching volume 7e52a442-8ea4-45e3-98eb-dc605d0d1d55 to /dev/vdb
Apr 12 03:36:22 user nova-compute[72081]: DEBUG os_brick.utils [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[30a6880a-a535-4e7e-85aa-04ddeb581663]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2acc3945-ae9e-497e-b380-fee63565d36f]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4f4cd92a-17ee-43b1-8ae1-e529f2229d99]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[32890213-926d-4e98-a1e0-e8d919f739e6]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:36:23 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:36:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:36:23 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:36:23 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6c7e0dc7-809b-4350-a81b-e614289da067]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:23 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:36:23 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:36:23 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:36:23 user nova-compute[72081]: DEBUG os_brick.utils [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] <== get_connector_properties: return (133ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:36:23 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Updating existing volume attachment record: a632a035-c19c-41d1-af2d-2531c818f883 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:36:23 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:36:23 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Took 0.63 seconds to deallocate network for instance.
Apr 12 03:36:23 user nova-compute[72081]: DEBUG nova.compute.manager [req-fbfd4e41-1a5c-4c68-9c9d-1323ef07e038 req-e0e6c5c8-7742-435c-b5e0-bf4e780cb871 service nova] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Received event network-vif-deleted-e3161de1-7643-4b1a-8a0c-27791fed8dbd {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:23 user nova-compute[72081]: INFO nova.compute.manager [req-fbfd4e41-1a5c-4c68-9c9d-1323ef07e038 req-e0e6c5c8-7742-435c-b5e0-bf4e780cb871 service nova] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Neutron deleted interface e3161de1-7643-4b1a-8a0c-27791fed8dbd; detaching it from the instance and deleting it from the info cache
Apr 12 03:36:23 user nova-compute[72081]: DEBUG nova.network.neutron [req-fbfd4e41-1a5c-4c68-9c9d-1323ef07e038 req-e0e6c5c8-7742-435c-b5e0-bf4e780cb871 service nova] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:36:23 user nova-compute[72081]: DEBUG nova.compute.manager [req-fbfd4e41-1a5c-4c68-9c9d-1323ef07e038 req-e0e6c5c8-7742-435c-b5e0-bf4e780cb871 service nova] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Detach interface failed, port_id=e3161de1-7643-4b1a-8a0c-27791fed8dbd, reason: Instance 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f could not be found. {{(pid=72081) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 12 03:36:23 user nova-compute[72081]: WARNING nova.volume.cinder [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Attachment e0bdb88c-d143-44b9-a8d9-a97f09d530f3 does not exist. Ignoring.: cinderclient.exceptions.NotFound: Volume attachment could not be found with filter: attachment_id = e0bdb88c-d143-44b9-a8d9-a97f09d530f3. (HTTP 404) (Request-ID: req-4cc94d5a-4916-4c44-9f34-8b0552afa7a3)
Apr 12 03:36:23 user nova-compute[72081]: INFO nova.compute.manager [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Took 0.16 seconds to detach 1 volumes for instance.
Apr 12 03:36:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:24 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:36:24 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:36:24 user nova-compute[72081]: DEBUG nova.compute.manager [req-5c52b2be-cbb0-4a0c-85f8-c323168021f4 req-2282191c-e603-4ac2-abb8-5dcf87dbb443 service nova] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Received event network-vif-plugged-e3161de1-7643-4b1a-8a0c-27791fed8dbd {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5c52b2be-cbb0-4a0c-85f8-c323168021f4 req-2282191c-e603-4ac2-abb8-5dcf87dbb443 service nova] Acquiring lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5c52b2be-cbb0-4a0c-85f8-c323168021f4 req-2282191c-e603-4ac2-abb8-5dcf87dbb443 service nova] Lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5c52b2be-cbb0-4a0c-85f8-c323168021f4 req-2282191c-e603-4ac2-abb8-5dcf87dbb443 service nova] Lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:24 user nova-compute[72081]: DEBUG nova.compute.manager [req-5c52b2be-cbb0-4a0c-85f8-c323168021f4 req-2282191c-e603-4ac2-abb8-5dcf87dbb443 service nova] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] No waiting events found dispatching network-vif-plugged-e3161de1-7643-4b1a-8a0c-27791fed8dbd {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:24 user nova-compute[72081]: WARNING nova.compute.manager [req-5c52b2be-cbb0-4a0c-85f8-c323168021f4 req-2282191c-e603-4ac2-abb8-5dcf87dbb443 service nova] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Received unexpected event network-vif-plugged-e3161de1-7643-4b1a-8a0c-27791fed8dbd for instance with vm_state deleted and task_state None.
Apr 12 03:36:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.481s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:24 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Deleted allocations for instance 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f
Apr 12 03:36:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a04442b-a51a-477f-a620-1ed173632a4d tempest-VolumesSnapshotTestJSON-1513599190 tempest-VolumesSnapshotTestJSON-1513599190-project-member] Lock "3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.586s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'yUzCJkbp', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:36:27 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:36:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:36:27 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:36:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:36:27 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5cfd6e2e-6ba3-458b-a6b5-693edcac9b79]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:36:27 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[11f11045-5c5a-49d0-a294-c5d9aa626db2]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55] added
Apr 12 03:36:27 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[267bdf3d-7553-4d75-a5c1-f8d5390a02be]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[409de16a-994b-4ef2-a564-0618fcd31d66]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v yUzCJkbp {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v yUzCJkbp" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5e4be5e5-4d69-4034-a9c4-0020e0983099]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'yUzCJkbp'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d65c6e86-354c-499f-b9d4-574f22615c85]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1cc381fa-5dbb-4646-a333-85153fa6d2f4]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:27 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:36:27 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:27 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:27 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:36:27 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:27 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --login" returned: 0 in 0.063s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[92de890b-4c82-47cd-b010-1e2e0a741e92]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55, portal: 172.16.0.220,3260]
Apr 12 03:36:27 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55, portal: 172.16.0.220,3260] successful.
Apr 12 03:36:27 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b6fe4382-f906-4630-bc5a-792c28a98034]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ae52fc84-01c8-44d8-afba-c295c3eef8c6]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)\ntcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:27 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:36:27 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:27 user nova-compute[72081]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:36:27 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:27 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:36:27 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:27 user nova-compute[72081]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:36:27 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.266s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] HCTL ('35', '-', '-', 0) found on session 20 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2ee771db-4bed-4c4a-956e-5f7a0725143f]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Searching for a device in session 20 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'yUzCJkbp', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:36:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.297s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:36:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] <== connect_volume: return (1299ms) {'type': 'block', 'scsi_wwn': '23563346662636263', 'path': '/dev/sdd'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:36:28 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23563346662636263', 'path': '/dev/sdd'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:36:28 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lazy-loading 'flavor' on Instance uuid 23367b24-0a3b-454d-901d-8a3e5396333a {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:36:28 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] attach device xml: <disk type="block" device="disk">
Apr 12 03:36:28 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:36:28 user nova-compute[72081]:   <source dev="/dev/sdd"/>
Apr 12 03:36:28 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:36:28 user nova-compute[72081]:   <serial>7e52a442-8ea4-45e3-98eb-dc605d0d1d55</serial>
Apr 12 03:36:28 user nova-compute[72081]: </disk>
Apr 12 03:36:28 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 03:36:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:36:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:36:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] No VIF found with MAC fa:16:3e:e4:18:3e, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:36:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9ab0c18f-14a4-4d83-8181-1fb3be87cfeb tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "23367b24-0a3b-454d-901d-8a3e5396333a" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.517s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-50fb0fc7-c9c8-4a0a-bf12-31f259389ab9 req-3f64045d-9188-45a4-a343-776dfb3f2c23 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Received event network-changed-a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-50fb0fc7-c9c8-4a0a-bf12-31f259389ab9 req-3f64045d-9188-45a4-a343-776dfb3f2c23 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Refreshing instance network info cache due to event network-changed-a76c6f07-234b-4832-8ace-ec81656b06a7. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:36:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-50fb0fc7-c9c8-4a0a-bf12-31f259389ab9 req-3f64045d-9188-45a4-a343-776dfb3f2c23 service nova] Acquiring lock "refresh_cache-23367b24-0a3b-454d-901d-8a3e5396333a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:36:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-50fb0fc7-c9c8-4a0a-bf12-31f259389ab9 req-3f64045d-9188-45a4-a343-776dfb3f2c23 service nova] Acquired lock "refresh_cache-23367b24-0a3b-454d-901d-8a3e5396333a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:36:33 user nova-compute[72081]: DEBUG nova.network.neutron [req-50fb0fc7-c9c8-4a0a-bf12-31f259389ab9 req-3f64045d-9188-45a4-a343-776dfb3f2c23 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Refreshing network info cache for port a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:36:34 user nova-compute[72081]: DEBUG nova.network.neutron [req-50fb0fc7-c9c8-4a0a-bf12-31f259389ab9 req-3f64045d-9188-45a4-a343-776dfb3f2c23 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Updated VIF entry in instance network info cache for port a76c6f07-234b-4832-8ace-ec81656b06a7. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:36:34 user nova-compute[72081]: DEBUG nova.network.neutron [req-50fb0fc7-c9c8-4a0a-bf12-31f259389ab9 req-3f64045d-9188-45a4-a343-776dfb3f2c23 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Updating instance_info_cache with network_info: [{"id": "a76c6f07-234b-4832-8ace-ec81656b06a7", "address": "fa:16:3e:e4:18:3e", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.29", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa76c6f07-23", "ovs_interfaceid": "a76c6f07-234b-4832-8ace-ec81656b06a7", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:36:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-50fb0fc7-c9c8-4a0a-bf12-31f259389ab9 req-3f64045d-9188-45a4-a343-776dfb3f2c23 service nova] Releasing lock "refresh_cache-23367b24-0a3b-454d-901d-8a3e5396333a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:36:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Acquiring lock "72ba1218-065d-40c4-8503-7c4155e47503" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "72ba1218-065d-40c4-8503-7c4155e47503" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:34 user nova-compute[72081]: INFO nova.compute.manager [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Detaching volume d2e08409-6c4e-44ef-842b-29460b0cb939
Apr 12 03:36:35 user nova-compute[72081]: INFO nova.virt.block_device [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Attempting to driver detach volume d2e08409-6c4e-44ef-842b-29460b0cb939 from mountpoint /dev/vdb
Apr 12 03:36:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Attempting to detach device vdb from instance 72ba1218-065d-40c4-8503-7c4155e47503 from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:36:35 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:36:35 user nova-compute[72081]:   <source dev="/dev/sdc"/>
Apr 12 03:36:35 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:36:35 user nova-compute[72081]:   <serial>d2e08409-6c4e-44ef-842b-29460b0cb939</serial>
Apr 12 03:36:35 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 03:36:35 user nova-compute[72081]: </disk>
Apr 12 03:36:35 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:36:35 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Successfully detached device vdb from instance 72ba1218-065d-40c4-8503-7c4155e47503 from the persistent domain config.
Apr 12 03:36:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 72ba1218-065d-40c4-8503-7c4155e47503 from the live domain config. {{(pid=72081) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:36:35 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:36:35 user nova-compute[72081]:   <source dev="/dev/sdc"/>
Apr 12 03:36:35 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:36:35 user nova-compute[72081]:   <serial>d2e08409-6c4e-44ef-842b-29460b0cb939</serial>
Apr 12 03:36:35 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 03:36:35 user nova-compute[72081]: </disk>
Apr 12 03:36:35 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Received event <DeviceRemovedEvent: 1681270595.5046196, 72ba1218-065d-40c4-8503-7c4155e47503 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72081) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 72ba1218-065d-40c4-8503-7c4155e47503 {{(pid=72081) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 12 03:36:35 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Successfully detached device vdb from instance 72ba1218-065d-40c4-8503-7c4155e47503 from the live domain config.
Apr 12 03:36:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'VLuEwIle', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:36:35 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:36:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0070a97c-729b-4e40-9209-244fe6d3ca3b]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:36:35 user nova-compute[72081]: No targets found.
Apr 12 03:36:35 user nova-compute[72081]: iSNS:
Apr 12 03:36:35 user nova-compute[72081]: No targets found.
Apr 12 03:36:35 user nova-compute[72081]: STATIC:
Apr 12 03:36:35 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:36:35 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:36:35 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:36:35 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f
Apr 12 03:36:35 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:36:35 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:36:35 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939
Apr 12 03:36:35 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:36:35 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:36:35 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55
Apr 12 03:36:35 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:36:35 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:36:35 user nova-compute[72081]: FIRMWARE:
Apr 12 03:36:35 user nova-compute[72081]: No targets found.
Apr 12 03:36:35 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:36:35 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:36:35 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[09b8bb12-4a0e-4bcd-b539-51d1c98709fc]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f0b16e2c-bdcb-47df-8a2a-411bf2c60e91]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)\ntcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:35 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:36:35 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:35 user nova-compute[72081]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:36:35 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:35 user nova-compute[72081]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 (non-flash)
Apr 12 03:36:35 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:35 user nova-compute[72081]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:36:35 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3f94130670>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939'): ({'sdc'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Removing single pathed devices sdc {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[72346a28-aed7-4439-a64f-9fe627203120]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[23b591c7-e815-432b-9bc1-1521e71686f2]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Flushing IO for device /dev/sdc {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.012s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8d5b0150-8c64-41bc-b6d2-9b06cad19d1e]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.046s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6c655b9c-0c35-4b47-a082-4a7ba4a6f7b7]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] SCSI volumes sdc have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG nova.compute.manager [req-3af399a6-9b92-48d7-8592-2488e35e2656 req-e99763f9-2154-4789-9a0f-106bd5b12831 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Received event network-changed-a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG nova.compute.manager [req-3af399a6-9b92-48d7-8592-2488e35e2656 req-e99763f9-2154-4789-9a0f-106bd5b12831 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Refreshing instance network info cache due to event network-changed-a76c6f07-234b-4832-8ace-ec81656b06a7. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3af399a6-9b92-48d7-8592-2488e35e2656 req-e99763f9-2154-4789-9a0f-106bd5b12831 service nova] Acquiring lock "refresh_cache-23367b24-0a3b-454d-901d-8a3e5396333a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3af399a6-9b92-48d7-8592-2488e35e2656 req-e99763f9-2154-4789-9a0f-106bd5b12831 service nova] Acquired lock "refresh_cache-23367b24-0a3b-454d-901d-8a3e5396333a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG nova.network.neutron [req-3af399a6-9b92-48d7-8592-2488e35e2656 req-e99763f9-2154-4789-9a0f-106bd5b12831 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Refreshing network info cache for port a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b73ddb04-cc1d-43c7-81fb-6958d360ef6a]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --logout" returned: 0 in 0.072s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e169040e-9b79-4186-aa7c-1d662b7254df]: (4, ('Logging out of session [sid: 15, target: iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939, portal: 172.16.0.220,3260]\nLogout of [sid: 15, target: iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 15, target: iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939, portal: 172.16.0.220,3260]
Apr 12 03:36:35 user nova-compute[72081]: Logout of [sid: 15, target: iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939, portal: 172.16.0.220,3260] successful.
Apr 12 03:36:35 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d2e08409-6c4e-44ef-842b-29460b0cb939 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[dd4b1def-c626-4b9f-b1a7-3725fef28437]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.321s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] <== disconnect_volume: return (322ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:36:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:36:36 user nova-compute[72081]: DEBUG nova.network.neutron [req-3af399a6-9b92-48d7-8592-2488e35e2656 req-e99763f9-2154-4789-9a0f-106bd5b12831 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Updated VIF entry in instance network info cache for port a76c6f07-234b-4832-8ace-ec81656b06a7. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:36:36 user nova-compute[72081]: DEBUG nova.network.neutron [req-3af399a6-9b92-48d7-8592-2488e35e2656 req-e99763f9-2154-4789-9a0f-106bd5b12831 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Updating instance_info_cache with network_info: [{"id": "a76c6f07-234b-4832-8ace-ec81656b06a7", "address": "fa:16:3e:e4:18:3e", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.29", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa76c6f07-23", "ovs_interfaceid": "a76c6f07-234b-4832-8ace-ec81656b06a7", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:36:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3af399a6-9b92-48d7-8592-2488e35e2656 req-e99763f9-2154-4789-9a0f-106bd5b12831 service nova] Releasing lock "refresh_cache-23367b24-0a3b-454d-901d-8a3e5396333a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:36:36 user nova-compute[72081]: DEBUG nova.objects.instance [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lazy-loading 'flavor' on Instance uuid 72ba1218-065d-40c4-8503-7c4155e47503 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:36:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2019e5fc-7e69-410f-93da-8c77d47ca69c tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "72ba1218-065d-40c4-8503-7c4155e47503" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.708s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-e7c1f03b-f492-436b-8677-93dce35b7007 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-e7c1f03b-f492-436b-8677-93dce35b7007 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-e7c1f03b-f492-436b-8677-93dce35b7007 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-e7c1f03b-f492-436b-8677-93dce35b7007 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-e7c1f03b-f492-436b-8677-93dce35b7007 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:37 user nova-compute[72081]: INFO nova.compute.manager [None req-e7c1f03b-f492-436b-8677-93dce35b7007 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Terminating instance
Apr 12 03:36:37 user nova-compute[72081]: DEBUG nova.compute.manager [None req-e7c1f03b-f492-436b-8677-93dce35b7007 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:36:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "23367b24-0a3b-454d-901d-8a3e5396333a" by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "23367b24-0a3b-454d-901d-8a3e5396333a" acquired by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:37 user nova-compute[72081]: INFO nova.compute.manager [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Rebooting instance
Apr 12 03:36:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "refresh_cache-23367b24-0a3b-454d-901d-8a3e5396333a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:36:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquired lock "refresh_cache-23367b24-0a3b-454d-901d-8a3e5396333a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:36:37 user nova-compute[72081]: DEBUG nova.network.neutron [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:36:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:37 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681270582.6163273, 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:36:37 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] VM Stopped (Lifecycle Event)
Apr 12 03:36:37 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8b2ec204-9098-4660-93dd-f3c6ffece476 None None] [instance: 3ea6c8a8-e6d1-4095-8b7b-5df7bf82c61f] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:36:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-428d2bc7-0656-4ff0-92e7-8da5a119bd98 req-6d5ec58f-76be-4895-b71e-c5e880a6de5e service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Received event network-vif-unplugged-540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-428d2bc7-0656-4ff0-92e7-8da5a119bd98 req-6d5ec58f-76be-4895-b71e-c5e880a6de5e service nova] Acquiring lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-428d2bc7-0656-4ff0-92e7-8da5a119bd98 req-6d5ec58f-76be-4895-b71e-c5e880a6de5e service nova] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-428d2bc7-0656-4ff0-92e7-8da5a119bd98 req-6d5ec58f-76be-4895-b71e-c5e880a6de5e service nova] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-428d2bc7-0656-4ff0-92e7-8da5a119bd98 req-6d5ec58f-76be-4895-b71e-c5e880a6de5e service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] No waiting events found dispatching network-vif-unplugged-540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-428d2bc7-0656-4ff0-92e7-8da5a119bd98 req-6d5ec58f-76be-4895-b71e-c5e880a6de5e service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Received event network-vif-unplugged-540b7307-8fea-4b50-bab8-626983135a96 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:36:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:38 user nova-compute[72081]: DEBUG nova.network.neutron [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Updating instance_info_cache with network_info: [{"id": "a76c6f07-234b-4832-8ace-ec81656b06a7", "address": "fa:16:3e:e4:18:3e", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.29", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa76c6f07-23", "ovs_interfaceid": "a76c6f07-234b-4832-8ace-ec81656b06a7", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:36:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Releasing lock "refresh_cache-23367b24-0a3b-454d-901d-8a3e5396333a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:36:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:36:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:38 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Instance destroyed successfully.
Apr 12 03:36:38 user nova-compute[72081]: DEBUG nova.objects.instance [None req-e7c1f03b-f492-436b-8677-93dce35b7007 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lazy-loading 'resources' on Instance uuid b23d4bcd-989a-45e1-89a3-6e3d30229a32 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:36:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-e7c1f03b-f492-436b-8677-93dce35b7007 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-12T03:30:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-1925048093',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-1925048093',id=3,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBNv3aMnfqZBrraTWMhLhVcBPnIqd/pyo9633J0szzOpRjklWBRcLk+TXv631ENNLeAHi6cKzIvn2qCEpBSN9HX63yWj5klT0kBwkXMpaNwu0eHxsmOThEKjvkK+5wRQIg==',key_name='tempest-keypair-892402082',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:34:18Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='738ebb6b798c4974aa46f9bbe3828279',ramdisk_id='',reservation_id='r-ybseqfps',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-2139975165',owner_user_name='tempest-AttachVolumeShelveTestJSON-2139975165-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:34:19Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='de1fe5db01624224ab7b615c908d990b',uuid=b23d4bcd-989a-45e1-89a3-6e3d30229a32,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "540b7307-8fea-4b50-bab8-626983135a96", "address": "fa:16:3e:8d:a9:ab", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.135", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap540b7307-8f", "ovs_interfaceid": "540b7307-8fea-4b50-bab8-626983135a96", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:36:38 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-e7c1f03b-f492-436b-8677-93dce35b7007 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Converting VIF {"id": "540b7307-8fea-4b50-bab8-626983135a96", "address": "fa:16:3e:8d:a9:ab", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.135", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap540b7307-8f", "ovs_interfaceid": "540b7307-8fea-4b50-bab8-626983135a96", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:36:38 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-e7c1f03b-f492-436b-8677-93dce35b7007 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:8d:a9:ab,bridge_name='br-int',has_traffic_filtering=True,id=540b7307-8fea-4b50-bab8-626983135a96,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap540b7307-8f') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:36:38 user nova-compute[72081]: DEBUG os_vif [None req-e7c1f03b-f492-436b-8677-93dce35b7007 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:8d:a9:ab,bridge_name='br-int',has_traffic_filtering=True,id=540b7307-8fea-4b50-bab8-626983135a96,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap540b7307-8f') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:36:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap540b7307-8f, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:36:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:38 user nova-compute[72081]: INFO os_vif [None req-e7c1f03b-f492-436b-8677-93dce35b7007 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:8d:a9:ab,bridge_name='br-int',has_traffic_filtering=True,id=540b7307-8fea-4b50-bab8-626983135a96,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap540b7307-8f')
Apr 12 03:36:38 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-e7c1f03b-f492-436b-8677-93dce35b7007 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Deleting instance files /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32_del
Apr 12 03:36:38 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-e7c1f03b-f492-436b-8677-93dce35b7007 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Deletion of /opt/stack/data/nova/instances/b23d4bcd-989a-45e1-89a3-6e3d30229a32_del complete
Apr 12 03:36:38 user nova-compute[72081]: INFO nova.compute.manager [None req-e7c1f03b-f492-436b-8677-93dce35b7007 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Took 0.86 seconds to destroy the instance on the hypervisor.
Apr 12 03:36:38 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-e7c1f03b-f492-436b-8677-93dce35b7007 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:36:38 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:36:38 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:36:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-260c10aa-6810-49aa-ab5c-ca38f8c04f4f req-11e81623-520f-4fed-a0f0-e1e40d1a0584 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Received event network-vif-unplugged-a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-260c10aa-6810-49aa-ab5c-ca38f8c04f4f req-11e81623-520f-4fed-a0f0-e1e40d1a0584 service nova] Acquiring lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-260c10aa-6810-49aa-ab5c-ca38f8c04f4f req-11e81623-520f-4fed-a0f0-e1e40d1a0584 service nova] Lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-260c10aa-6810-49aa-ab5c-ca38f8c04f4f req-11e81623-520f-4fed-a0f0-e1e40d1a0584 service nova] Lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-260c10aa-6810-49aa-ab5c-ca38f8c04f4f req-11e81623-520f-4fed-a0f0-e1e40d1a0584 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] No waiting events found dispatching network-vif-unplugged-a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:38 user nova-compute[72081]: WARNING nova.compute.manager [req-260c10aa-6810-49aa-ab5c-ca38f8c04f4f req-11e81623-520f-4fed-a0f0-e1e40d1a0584 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Received unexpected event network-vif-unplugged-a76c6f07-234b-4832-8ace-ec81656b06a7 for instance with vm_state active and task_state reboot_started_hard.
Apr 12 03:36:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:39 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Instance destroyed successfully.
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.objects.instance [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lazy-loading 'resources' on Instance uuid 23367b24-0a3b-454d-901d-8a3e5396333a {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:34:29Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-2135071440',display_name='tempest-TestMinimumBasicScenario-server-2135071440',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-2135071440',id=17,image_ref='5267358d-2283-43f2-a893-8842d5de70da',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMkexWkk4Rxu6k02iex2AsbJ+PDY5htmdDN8vQSba2S3zwrjAPkO8wlI5tbs0OLJWNWhrMe6/AoKb3eoA7zRDJCJlh8REZk8FLy3CbfuwbBsBPrnYnRJ2rfKKtaAGF256w==',key_name='tempest-TestMinimumBasicScenario-2058975683',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:34:38Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='05e1e25933cd42df98fb060b8d2701dc',ramdisk_id='',reservation_id='r-0n9gnxqj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='5267358d-2283-43f2-a893-8842d5de70da',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-TestMinimumBasicScenario-1605347971',owner_user_name='tempest-TestMinimumBasicScenario-1605347971-project-member'},tags=<?>,task_state='reboot_started_hard',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:36:38Z,user_data=None,user_id='3f8024260bf14d9995e67cb0d1bc8ddc',uuid=23367b24-0a3b-454d-901d-8a3e5396333a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "a76c6f07-234b-4832-8ace-ec81656b06a7", "address": "fa:16:3e:e4:18:3e", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.29", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa76c6f07-23", "ovs_interfaceid": "a76c6f07-234b-4832-8ace-ec81656b06a7", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Converting VIF {"id": "a76c6f07-234b-4832-8ace-ec81656b06a7", "address": "fa:16:3e:e4:18:3e", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.29", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa76c6f07-23", "ovs_interfaceid": "a76c6f07-234b-4832-8ace-ec81656b06a7", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:e4:18:3e,bridge_name='br-int',has_traffic_filtering=True,id=a76c6f07-234b-4832-8ace-ec81656b06a7,network=Network(91a7501a-44a5-4d48-a1f7-e1195ac99607),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa76c6f07-23') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_vif [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:e4:18:3e,bridge_name='br-int',has_traffic_filtering=True,id=a76c6f07-234b-4832-8ace-ec81656b06a7,network=Network(91a7501a-44a5-4d48-a1f7-e1195ac99607),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa76c6f07-23') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapa76c6f07-23, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:36:39 user nova-compute[72081]: INFO os_vif [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:e4:18:3e,bridge_name='br-int',has_traffic_filtering=True,id=a76c6f07-234b-4832-8ace-ec81656b06a7,network=Network(91a7501a-44a5-4d48-a1f7-e1195ac99607),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa76c6f07-23')
Apr 12 03:36:39 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Took 0.77 seconds to deallocate network for instance.
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'yUzCJkbp', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:36:39 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.compute.manager [req-e3c49aa7-dbd7-40f5-b831-7a403f14486c req-48e6df8e-9187-4f5e-9481-ff3b65b5a7d8 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Received event network-vif-deleted-540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:39 user nova-compute[72081]: INFO nova.compute.manager [req-e3c49aa7-dbd7-40f5-b831-7a403f14486c req-48e6df8e-9187-4f5e-9481-ff3b65b5a7d8 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Neutron deleted interface 540b7307-8fea-4b50-bab8-626983135a96; detaching it from the instance and deleting it from the info cache
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.network.neutron [req-e3c49aa7-dbd7-40f5-b831-7a403f14486c req-48e6df8e-9187-4f5e-9481-ff3b65b5a7d8 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ee636de9-bea6-4cf7-8d80-a2843b93dcf0]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:36:39 user nova-compute[72081]: No targets found.
Apr 12 03:36:39 user nova-compute[72081]: iSNS:
Apr 12 03:36:39 user nova-compute[72081]: No targets found.
Apr 12 03:36:39 user nova-compute[72081]: STATIC:
Apr 12 03:36:39 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:36:39 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:36:39 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:36:39 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f
Apr 12 03:36:39 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:36:39 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:36:39 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55
Apr 12 03:36:39 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:36:39 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:36:39 user nova-compute[72081]: FIRMWARE:
Apr 12 03:36:39 user nova-compute[72081]: No targets found.
Apr 12 03:36:39 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:36:39 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:36:39 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.compute.manager [req-e3c49aa7-dbd7-40f5-b831-7a403f14486c req-48e6df8e-9187-4f5e-9481-ff3b65b5a7d8 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Detach interface failed, port_id=540b7307-8fea-4b50-bab8-626983135a96, reason: Instance b23d4bcd-989a-45e1-89a3-6e3d30229a32 could not be found. {{(pid=72081) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bc8eb675-b633-4e24-b67b-b939bbe1e104]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-e7c1f03b-f492-436b-8677-93dce35b7007 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-e7c1f03b-f492-436b-8677-93dce35b7007 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[be6d51eb-1698-44fd-9c4c-b9fcdf21b4dc]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)\ntcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:39 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:39 user nova-compute[72081]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:36:39 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:39 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:39 user nova-compute[72081]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:36:39 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3f94132200>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55'): ({'sdd'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Removing single pathed devices sdd {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[99a62e47-8f36-40d0-bcfe-eaa767711b52]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c37ff8e-f1be-4472-b1fa-80b2f886a01c]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Flushing IO for device /dev/sdd {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[43e7f13b-7843-4466-b170-26d4aa5f375e]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.044s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e6df66ea-7ae3-49fb-9655-cc5bec1352e2]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] SCSI volumes sdd have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c25525bc-73ce-4eac-839e-4235fe4c6693]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --logout" returned: 0 in 0.063s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4e5217f3-a178-4628-aa06-eb2de956c472]: (4, ('Logging out of session [sid: 20, target: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55, portal: 172.16.0.220,3260]\nLogout of [sid: 20, target: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 20, target: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55, portal: 172.16.0.220,3260]
Apr 12 03:36:39 user nova-compute[72081]: Logout of [sid: 20, target: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55, portal: 172.16.0.220,3260] successful.
Apr 12 03:36:39 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d069c75-82bc-4b60-997d-411e9927221e]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.309s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] <== disconnect_volume: return (310ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Start _get_guest_xml network_info=[{"id": "a76c6f07-234b-4832-8ace-ec81656b06a7", "address": "fa:16:3e:e4:18:3e", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.29", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa76c6f07-23", "ovs_interfaceid": "a76c6f07-234b-4832-8ace-ec81656b06a7", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=5267358d-2283-43f2-a893-8842d5de70da,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': '5267358d-2283-43f2-a893-8842d5de70da'}], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vdb', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'yUzCJkbp', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd'}, 'status': 'reserved', 'instance': '23367b24-0a3b-454d-901d-8a3e5396333a', 'attached_at': '', 'detached_at': '', 'volume_id': '7e52a442-8ea4-45e3-98eb-dc605d0d1d55', 'serial': '7e52a442-8ea4-45e3-98eb-dc605d0d1d55'}, 'disk_bus': 'virtio', 'delete_on_termination': False, 'boot_index': None, 'guest_format': None, 'device_type': 'disk', 'attachment_id': 'a632a035-c19c-41d1-af2d-2531c818f883', 'volume_type': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:36:39 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:36:39 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=5267358d-2283-43f2-a893-8842d5de70da,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.objects.instance [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lazy-loading 'vcpu_model' on Instance uuid 23367b24-0a3b-454d-901d-8a3e5396333a {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-e7c1f03b-f492-436b-8677-93dce35b7007 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'yUzCJkbp', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:36:39 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:36:39 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:36:39 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-e7c1f03b-f492-436b-8677-93dce35b7007 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[293420d7-a215-434a-93b6-b091e97ab297]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:36:39 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-e7c1f03b-f492-436b-8677-93dce35b7007 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.366s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0376b310-d890-4b7d-b8e0-97127c14be77]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55] added
Apr 12 03:36:39 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1f807ece-54f3-4d02-9507-118c86e9d5f3]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:39 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-e7c1f03b-f492-436b-8677-93dce35b7007 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Deleted allocations for instance b23d4bcd-989a-45e1-89a3-6e3d30229a32
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b788e80e-bd2d-450a-8eaa-9038cb85d18b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v yUzCJkbp {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v yUzCJkbp" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2a730c8b-0754-42bc-9106-66f8612dc07b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'yUzCJkbp'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2f4ae320-e564-4072-bbeb-694ddc97fa62]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-e7c1f03b-f492-436b-8677-93dce35b7007 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.177s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a239eb8c-ed00-441f-ade6-f31aeb3e9839]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:39 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:39 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:39 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:39 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --login" returned: 0 in 0.062s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[19b5f26e-cb95-4ccf-a11a-107f08940f62]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55, portal: 172.16.0.220,3260]
Apr 12 03:36:39 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55, portal: 172.16.0.220,3260] successful.
Apr 12 03:36:39 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f1f40b08-96f5-4e5f-a8a1-b6f4ac4809d5]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2a3654e6-c1e5-4574-9ccb-307ea34b0f1b]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:39 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:39 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:36:39 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:39 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:39 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:36:39 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.265s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] HCTL ('34', '-', '-', 0) found on session 21 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d48c0336-4632-4162-ace8-7fcf0a6522e1]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Searching for a device in session 21 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'yUzCJkbp', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.compute.manager [req-62a366f3-ed95-497b-9a80-194e84797f7f req-e6322069-3e38-4ddf-a213-7e2bc7431fd7 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Received event network-vif-plugged-540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-62a366f3-ed95-497b-9a80-194e84797f7f req-e6322069-3e38-4ddf-a213-7e2bc7431fd7 service nova] Acquiring lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-62a366f3-ed95-497b-9a80-194e84797f7f req-e6322069-3e38-4ddf-a213-7e2bc7431fd7 service nova] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-62a366f3-ed95-497b-9a80-194e84797f7f req-e6322069-3e38-4ddf-a213-7e2bc7431fd7 service nova] Lock "b23d4bcd-989a-45e1-89a3-6e3d30229a32-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:39 user nova-compute[72081]: DEBUG nova.compute.manager [req-62a366f3-ed95-497b-9a80-194e84797f7f req-e6322069-3e38-4ddf-a213-7e2bc7431fd7 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] No waiting events found dispatching network-vif-plugged-540b7307-8fea-4b50-bab8-626983135a96 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:39 user nova-compute[72081]: WARNING nova.compute.manager [req-62a366f3-ed95-497b-9a80-194e84797f7f req-e6322069-3e38-4ddf-a213-7e2bc7431fd7 service nova] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Received unexpected event network-vif-plugged-540b7307-8fea-4b50-bab8-626983135a96 for instance with vm_state deleted and task_state None.
Apr 12 03:36:40 user nova-compute[72081]: DEBUG nova.compute.manager [req-11ba383e-d33c-411f-877b-4ed58074b6e9 req-c01355dc-1a05-459b-b28d-fe02b8d2d319 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Received event network-vif-plugged-a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-11ba383e-d33c-411f-877b-4ed58074b6e9 req-c01355dc-1a05-459b-b28d-fe02b8d2d319 service nova] Acquiring lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-11ba383e-d33c-411f-877b-4ed58074b6e9 req-c01355dc-1a05-459b-b28d-fe02b8d2d319 service nova] Lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-11ba383e-d33c-411f-877b-4ed58074b6e9 req-c01355dc-1a05-459b-b28d-fe02b8d2d319 service nova] Lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:40 user nova-compute[72081]: DEBUG nova.compute.manager [req-11ba383e-d33c-411f-877b-4ed58074b6e9 req-c01355dc-1a05-459b-b28d-fe02b8d2d319 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] No waiting events found dispatching network-vif-plugged-a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:40 user nova-compute[72081]: WARNING nova.compute.manager [req-11ba383e-d33c-411f-877b-4ed58074b6e9 req-c01355dc-1a05-459b-b28d-fe02b8d2d319 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Received unexpected event network-vif-plugged-a76c6f07-234b-4832-8ace-ec81656b06a7 for instance with vm_state active and task_state reboot_started_hard.
Apr 12 03:36:40 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.294s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:36:40 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] <== connect_volume: return (1296ms) {'type': 'block', 'scsi_wwn': '23563346662636263', 'path': '/dev/sdc'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:36:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23563346662636263', 'path': '/dev/sdc'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:36:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:34:29Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-2135071440',display_name='tempest-TestMinimumBasicScenario-server-2135071440',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-2135071440',id=17,image_ref='5267358d-2283-43f2-a893-8842d5de70da',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMkexWkk4Rxu6k02iex2AsbJ+PDY5htmdDN8vQSba2S3zwrjAPkO8wlI5tbs0OLJWNWhrMe6/AoKb3eoA7zRDJCJlh8REZk8FLy3CbfuwbBsBPrnYnRJ2rfKKtaAGF256w==',key_name='tempest-TestMinimumBasicScenario-2058975683',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:34:38Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='05e1e25933cd42df98fb060b8d2701dc',ramdisk_id='',reservation_id='r-0n9gnxqj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='5267358d-2283-43f2-a893-8842d5de70da',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-TestMinimumBasicScenario-1605347971',owner_user_name='tempest-TestMinimumBasicScenario-1605347971-project-member'},tags=<?>,task_state='reboot_started_hard',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:36:38Z,user_data=None,user_id='3f8024260bf14d9995e67cb0d1bc8ddc',uuid=23367b24-0a3b-454d-901d-8a3e5396333a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "a76c6f07-234b-4832-8ace-ec81656b06a7", "address": "fa:16:3e:e4:18:3e", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.29", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa76c6f07-23", "ovs_interfaceid": "a76c6f07-234b-4832-8ace-ec81656b06a7", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:36:40 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Converting VIF {"id": "a76c6f07-234b-4832-8ace-ec81656b06a7", "address": "fa:16:3e:e4:18:3e", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.29", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa76c6f07-23", "ovs_interfaceid": "a76c6f07-234b-4832-8ace-ec81656b06a7", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:36:40 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:e4:18:3e,bridge_name='br-int',has_traffic_filtering=True,id=a76c6f07-234b-4832-8ace-ec81656b06a7,network=Network(91a7501a-44a5-4d48-a1f7-e1195ac99607),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa76c6f07-23') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:36:40 user nova-compute[72081]: DEBUG nova.objects.instance [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lazy-loading 'pci_devices' on Instance uuid 23367b24-0a3b-454d-901d-8a3e5396333a {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:36:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:36:40 user nova-compute[72081]:   <uuid>23367b24-0a3b-454d-901d-8a3e5396333a</uuid>
Apr 12 03:36:40 user nova-compute[72081]:   <name>instance-00000011</name>
Apr 12 03:36:40 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:36:40 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:36:40 user nova-compute[72081]:   <metadata>
Apr 12 03:36:40 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:36:40 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:36:40 user nova-compute[72081]:       <nova:name>tempest-TestMinimumBasicScenario-server-2135071440</nova:name>
Apr 12 03:36:40 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:36:39</nova:creationTime>
Apr 12 03:36:40 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:36:40 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:36:40 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:36:40 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:36:40 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:36:40 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:36:40 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:36:40 user nova-compute[72081]:       <nova:owner>
Apr 12 03:36:40 user nova-compute[72081]:         <nova:user uuid="3f8024260bf14d9995e67cb0d1bc8ddc">tempest-TestMinimumBasicScenario-1605347971-project-member</nova:user>
Apr 12 03:36:40 user nova-compute[72081]:         <nova:project uuid="05e1e25933cd42df98fb060b8d2701dc">tempest-TestMinimumBasicScenario-1605347971</nova:project>
Apr 12 03:36:40 user nova-compute[72081]:       </nova:owner>
Apr 12 03:36:40 user nova-compute[72081]:       <nova:root type="image" uuid="5267358d-2283-43f2-a893-8842d5de70da"/>
Apr 12 03:36:40 user nova-compute[72081]:       <nova:ports>
Apr 12 03:36:40 user nova-compute[72081]:         <nova:port uuid="a76c6f07-234b-4832-8ace-ec81656b06a7">
Apr 12 03:36:40 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 12 03:36:40 user nova-compute[72081]:         </nova:port>
Apr 12 03:36:40 user nova-compute[72081]:       </nova:ports>
Apr 12 03:36:40 user nova-compute[72081]:     </nova:instance>
Apr 12 03:36:40 user nova-compute[72081]:   </metadata>
Apr 12 03:36:40 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:36:40 user nova-compute[72081]:     <system>
Apr 12 03:36:40 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:36:40 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:36:40 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:36:40 user nova-compute[72081]:       <entry name="serial">23367b24-0a3b-454d-901d-8a3e5396333a</entry>
Apr 12 03:36:40 user nova-compute[72081]:       <entry name="uuid">23367b24-0a3b-454d-901d-8a3e5396333a</entry>
Apr 12 03:36:40 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:36:40 user nova-compute[72081]:     </system>
Apr 12 03:36:40 user nova-compute[72081]:   </sysinfo>
Apr 12 03:36:40 user nova-compute[72081]:   <os>
Apr 12 03:36:40 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:36:40 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:36:40 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:36:40 user nova-compute[72081]:   </os>
Apr 12 03:36:40 user nova-compute[72081]:   <features>
Apr 12 03:36:40 user nova-compute[72081]:     <acpi/>
Apr 12 03:36:40 user nova-compute[72081]:     <apic/>
Apr 12 03:36:40 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:36:40 user nova-compute[72081]:   </features>
Apr 12 03:36:40 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:36:40 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:36:40 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:36:40 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:36:40 user nova-compute[72081]:   </clock>
Apr 12 03:36:40 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:36:40 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:36:40 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:36:40 user nova-compute[72081]:   </cpu>
Apr 12 03:36:40 user nova-compute[72081]:   <devices>
Apr 12 03:36:40 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:36:40 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:36:40 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk"/>
Apr 12 03:36:40 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:36:40 user nova-compute[72081]:     </disk>
Apr 12 03:36:40 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:36:40 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:36:40 user nova-compute[72081]:       <source dev="/dev/sdc"/>
Apr 12 03:36:40 user nova-compute[72081]:       <target dev="vdb" bus="virtio"/>
Apr 12 03:36:40 user nova-compute[72081]:       <serial>7e52a442-8ea4-45e3-98eb-dc605d0d1d55</serial>
Apr 12 03:36:40 user nova-compute[72081]:     </disk>
Apr 12 03:36:40 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:36:40 user nova-compute[72081]:       <mac address="fa:16:3e:e4:18:3e"/>
Apr 12 03:36:40 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:36:40 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:36:40 user nova-compute[72081]:       <target dev="tapa76c6f07-23"/>
Apr 12 03:36:40 user nova-compute[72081]:     </interface>
Apr 12 03:36:40 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:36:40 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/console.log" append="off"/>
Apr 12 03:36:40 user nova-compute[72081]:     </serial>
Apr 12 03:36:40 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:36:40 user nova-compute[72081]:     <video>
Apr 12 03:36:40 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:36:40 user nova-compute[72081]:     </video>
Apr 12 03:36:40 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:36:40 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:36:40 user nova-compute[72081]:     </rng>
Apr 12 03:36:40 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:36:40 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:36:40 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:36:40 user nova-compute[72081]:     </memballoon>
Apr 12 03:36:40 user nova-compute[72081]:   </devices>
Apr 12 03:36:40 user nova-compute[72081]: </domain>
Apr 12 03:36:40 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:36:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:41 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:41 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:36:41 user nova-compute[72081]: DEBUG nova.objects.instance [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lazy-loading 'trusted_certs' on Instance uuid 23367b24-0a3b-454d-901d-8a3e5396333a {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:36:41 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/20a9ef7c61ef4dc87afe749fc5ad5499efda7617 --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:41 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/20a9ef7c61ef4dc87afe749fc5ad5499efda7617 --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:41 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Checking if we can resize image /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:36:41 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:41 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:41 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Cannot resize image /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:36:41 user nova-compute[72081]: DEBUG nova.objects.instance [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lazy-loading 'migration_context' on Instance uuid 23367b24-0a3b-454d-901d-8a3e5396333a {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:36:41 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:34:29Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-2135071440',display_name='tempest-TestMinimumBasicScenario-server-2135071440',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-2135071440',id=17,image_ref='5267358d-2283-43f2-a893-8842d5de70da',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMkexWkk4Rxu6k02iex2AsbJ+PDY5htmdDN8vQSba2S3zwrjAPkO8wlI5tbs0OLJWNWhrMe6/AoKb3eoA7zRDJCJlh8REZk8FLy3CbfuwbBsBPrnYnRJ2rfKKtaAGF256w==',key_name='tempest-TestMinimumBasicScenario-2058975683',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:34:38Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=<?>,power_state=1,progress=0,project_id='05e1e25933cd42df98fb060b8d2701dc',ramdisk_id='',reservation_id='r-0n9gnxqj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='5267358d-2283-43f2-a893-8842d5de70da',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-TestMinimumBasicScenario-1605347971',owner_user_name='tempest-TestMinimumBasicScenario-1605347971-project-member'},tags=<?>,task_state='reboot_started_hard',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:36:38Z,user_data=None,user_id='3f8024260bf14d9995e67cb0d1bc8ddc',uuid=23367b24-0a3b-454d-901d-8a3e5396333a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "a76c6f07-234b-4832-8ace-ec81656b06a7", "address": "fa:16:3e:e4:18:3e", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.29", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa76c6f07-23", "ovs_interfaceid": "a76c6f07-234b-4832-8ace-ec81656b06a7", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:36:41 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Converting VIF {"id": "a76c6f07-234b-4832-8ace-ec81656b06a7", "address": "fa:16:3e:e4:18:3e", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.29", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa76c6f07-23", "ovs_interfaceid": "a76c6f07-234b-4832-8ace-ec81656b06a7", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:36:41 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:e4:18:3e,bridge_name='br-int',has_traffic_filtering=True,id=a76c6f07-234b-4832-8ace-ec81656b06a7,network=Network(91a7501a-44a5-4d48-a1f7-e1195ac99607),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa76c6f07-23') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:36:41 user nova-compute[72081]: DEBUG os_vif [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Plugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:e4:18:3e,bridge_name='br-int',has_traffic_filtering=True,id=a76c6f07-234b-4832-8ace-ec81656b06a7,network=Network(91a7501a-44a5-4d48-a1f7-e1195ac99607),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa76c6f07-23') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:36:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:36:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:36:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapa76c6f07-23, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:36:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapa76c6f07-23, col_values=(('external_ids', {'iface-id': 'a76c6f07-234b-4832-8ace-ec81656b06a7', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:e4:18:3e', 'vm-uuid': '23367b24-0a3b-454d-901d-8a3e5396333a'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:36:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:36:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:41 user nova-compute[72081]: INFO os_vif [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Successfully plugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:e4:18:3e,bridge_name='br-int',has_traffic_filtering=True,id=a76c6f07-234b-4832-8ace-ec81656b06a7,network=Network(91a7501a-44a5-4d48-a1f7-e1195ac99607),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa76c6f07-23')
Apr 12 03:36:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:42 user nova-compute[72081]: DEBUG nova.compute.manager [req-b2d56ad0-c24a-4b26-b7cc-c664fe7abd64 req-48a7ef7b-3aa3-48cf-911c-b7dcc2bcdc3c service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Received event network-vif-plugged-a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b2d56ad0-c24a-4b26-b7cc-c664fe7abd64 req-48a7ef7b-3aa3-48cf-911c-b7dcc2bcdc3c service nova] Acquiring lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b2d56ad0-c24a-4b26-b7cc-c664fe7abd64 req-48a7ef7b-3aa3-48cf-911c-b7dcc2bcdc3c service nova] Lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b2d56ad0-c24a-4b26-b7cc-c664fe7abd64 req-48a7ef7b-3aa3-48cf-911c-b7dcc2bcdc3c service nova] Lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:42 user nova-compute[72081]: DEBUG nova.compute.manager [req-b2d56ad0-c24a-4b26-b7cc-c664fe7abd64 req-48a7ef7b-3aa3-48cf-911c-b7dcc2bcdc3c service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] No waiting events found dispatching network-vif-plugged-a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:42 user nova-compute[72081]: WARNING nova.compute.manager [req-b2d56ad0-c24a-4b26-b7cc-c664fe7abd64 req-48a7ef7b-3aa3-48cf-911c-b7dcc2bcdc3c service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Received unexpected event network-vif-plugged-a76c6f07-234b-4832-8ace-ec81656b06a7 for instance with vm_state active and task_state reboot_started_hard.
Apr 12 03:36:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:44 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Removed pending event for 23367b24-0a3b-454d-901d-8a3e5396333a due to event {{(pid=72081) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 12 03:36:44 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270604.7703774, 23367b24-0a3b-454d-901d-8a3e5396333a => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:36:44 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] VM Resumed (Lifecycle Event)
Apr 12 03:36:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:36:44 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Instance rebooted successfully.
Apr 12 03:36:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:36:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:36:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: reboot_started_hard, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:36:44 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] During sync_power_state the instance has a pending task (reboot_started_hard). Skip.
Apr 12 03:36:44 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270604.7719631, 23367b24-0a3b-454d-901d-8a3e5396333a => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:36:44 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] VM Started (Lifecycle Event)
Apr 12 03:36:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:36:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1622b8a6-1a3d-4f28-8e23-8f2a856ba007 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "23367b24-0a3b-454d-901d-8a3e5396333a" "released" by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" :: held 7.352s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: reboot_started_hard, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:36:45 user nova-compute[72081]: DEBUG nova.compute.manager [req-e08fe44b-4274-4900-9503-795eb57fbaa7 req-da2ea682-8bc1-4299-8fd0-df2ec962664d service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Received event network-vif-plugged-a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e08fe44b-4274-4900-9503-795eb57fbaa7 req-da2ea682-8bc1-4299-8fd0-df2ec962664d service nova] Acquiring lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e08fe44b-4274-4900-9503-795eb57fbaa7 req-da2ea682-8bc1-4299-8fd0-df2ec962664d service nova] Lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e08fe44b-4274-4900-9503-795eb57fbaa7 req-da2ea682-8bc1-4299-8fd0-df2ec962664d service nova] Lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:45 user nova-compute[72081]: DEBUG nova.compute.manager [req-e08fe44b-4274-4900-9503-795eb57fbaa7 req-da2ea682-8bc1-4299-8fd0-df2ec962664d service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] No waiting events found dispatching network-vif-plugged-a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:45 user nova-compute[72081]: WARNING nova.compute.manager [req-e08fe44b-4274-4900-9503-795eb57fbaa7 req-da2ea682-8bc1-4299-8fd0-df2ec962664d service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Received unexpected event network-vif-plugged-a76c6f07-234b-4832-8ace-ec81656b06a7 for instance with vm_state active and task_state None.
Apr 12 03:36:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_power_states {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Triggering sync for uuid 56834002-7593-4592-a86d-23bb90d00be0 {{(pid=72081) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Triggering sync for uuid 2d0904c3-74e0-47bb-a832-758a7035b2f5 {{(pid=72081) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Triggering sync for uuid 72ba1218-065d-40c4-8503-7c4155e47503 {{(pid=72081) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Triggering sync for uuid bcd267d2-5b2c-4ed2-b634-0210fae94dfd {{(pid=72081) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Triggering sync for uuid 48977eb2-ce43-4577-a1cb-340e35703dbd {{(pid=72081) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Triggering sync for uuid 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 {{(pid=72081) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Triggering sync for uuid 23367b24-0a3b-454d-901d-8a3e5396333a {{(pid=72081) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Triggering sync for uuid 49d33636-1dbb-4863-b0f3-9f1578e23e67 {{(pid=72081) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Triggering sync for uuid a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6 {{(pid=72081) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "56834002-7593-4592-a86d-23bb90d00be0" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "56834002-7593-4592-a86d-23bb90d00be0" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "2d0904c3-74e0-47bb-a832-758a7035b2f5" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "2d0904c3-74e0-47bb-a832-758a7035b2f5" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.004s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "72ba1218-065d-40c4-8503-7c4155e47503" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "72ba1218-065d-40c4-8503-7c4155e47503" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "23367b24-0a3b-454d-901d-8a3e5396333a" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "23367b24-0a3b-454d-901d-8a3e5396333a" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "49d33636-1dbb-4863-b0f3-9f1578e23e67" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "49d33636-1dbb-4863-b0f3-9f1578e23e67" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.004s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "23367b24-0a3b-454d-901d-8a3e5396333a" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.106s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "72ba1218-065d-40c4-8503-7c4155e47503" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.114s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "2d0904c3-74e0-47bb-a832-758a7035b2f5" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.116s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "56834002-7593-4592-a86d-23bb90d00be0" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.131s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.125s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.120s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.124s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.115s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "49d33636-1dbb-4863-b0f3-9f1578e23e67" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.123s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:51 user nova-compute[72081]: INFO nova.compute.manager [None req-c84c9787-e70c-4934-b03d-0e2c4ef93822 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Unrescuing
Apr 12 03:36:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c84c9787-e70c-4934-b03d-0e2c4ef93822 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "refresh_cache-bcd267d2-5b2c-4ed2-b634-0210fae94dfd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:36:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c84c9787-e70c-4934-b03d-0e2c4ef93822 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquired lock "refresh_cache-bcd267d2-5b2c-4ed2-b634-0210fae94dfd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:36:51 user nova-compute[72081]: DEBUG nova.network.neutron [None req-c84c9787-e70c-4934-b03d-0e2c4ef93822 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:36:52 user nova-compute[72081]: DEBUG nova.network.neutron [None req-c84c9787-e70c-4934-b03d-0e2c4ef93822 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Updating instance_info_cache with network_info: [{"id": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "address": "fa:16:3e:9d:0c:c2", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa4385731-05", "ovs_interfaceid": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:36:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c84c9787-e70c-4934-b03d-0e2c4ef93822 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Releasing lock "refresh_cache-bcd267d2-5b2c-4ed2-b634-0210fae94dfd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:36:52 user nova-compute[72081]: DEBUG nova.objects.instance [None req-c84c9787-e70c-4934-b03d-0e2c4ef93822 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lazy-loading 'flavor' on Instance uuid bcd267d2-5b2c-4ed2-b634-0210fae94dfd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:36:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:52 user nova-compute[72081]: DEBUG nova.compute.manager [req-f062e545-4dcb-4653-861c-05d336d5e362 req-b8c647ed-224d-4f56-93e7-013335d7636e service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received event network-vif-unplugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f062e545-4dcb-4653-861c-05d336d5e362 req-b8c647ed-224d-4f56-93e7-013335d7636e service nova] Acquiring lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f062e545-4dcb-4653-861c-05d336d5e362 req-b8c647ed-224d-4f56-93e7-013335d7636e service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f062e545-4dcb-4653-861c-05d336d5e362 req-b8c647ed-224d-4f56-93e7-013335d7636e service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:52 user nova-compute[72081]: DEBUG nova.compute.manager [req-f062e545-4dcb-4653-861c-05d336d5e362 req-b8c647ed-224d-4f56-93e7-013335d7636e service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] No waiting events found dispatching network-vif-unplugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:52 user nova-compute[72081]: WARNING nova.compute.manager [req-f062e545-4dcb-4653-861c-05d336d5e362 req-b8c647ed-224d-4f56-93e7-013335d7636e service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received unexpected event network-vif-unplugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 for instance with vm_state rescued and task_state unrescuing.
Apr 12 03:36:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:52 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Instance destroyed successfully.
Apr 12 03:36:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:53 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681270598.1710243, b23d4bcd-989a-45e1-89a3-6e3d30229a32 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:36:53 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] VM Stopped (Lifecycle Event)
Apr 12 03:36:53 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f1804831-bfb3-4bab-8012-b93f6123a95c None None] [instance: b23d4bcd-989a-45e1-89a3-6e3d30229a32] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:36:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG nova.compute.manager [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] Acquiring lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG nova.compute.manager [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] No waiting events found dispatching network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:54 user nova-compute[72081]: WARNING nova.compute.manager [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received unexpected event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 for instance with vm_state rescued and task_state unrescuing.
Apr 12 03:36:54 user nova-compute[72081]: DEBUG nova.compute.manager [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] Acquiring lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG nova.compute.manager [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] No waiting events found dispatching network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:54 user nova-compute[72081]: WARNING nova.compute.manager [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received unexpected event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 for instance with vm_state rescued and task_state unrescuing.
Apr 12 03:36:54 user nova-compute[72081]: DEBUG nova.compute.manager [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] Acquiring lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG nova.compute.manager [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] No waiting events found dispatching network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:54 user nova-compute[72081]: WARNING nova.compute.manager [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received unexpected event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 for instance with vm_state rescued and task_state unrescuing.
Apr 12 03:36:54 user nova-compute[72081]: DEBUG nova.compute.manager [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received event network-vif-unplugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] Acquiring lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG nova.compute.manager [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] No waiting events found dispatching network-vif-unplugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:54 user nova-compute[72081]: WARNING nova.compute.manager [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received unexpected event network-vif-unplugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 for instance with vm_state rescued and task_state unrescuing.
Apr 12 03:36:54 user nova-compute[72081]: DEBUG nova.compute.manager [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] Acquiring lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG nova.compute.manager [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] No waiting events found dispatching network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:54 user nova-compute[72081]: WARNING nova.compute.manager [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received unexpected event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 for instance with vm_state rescued and task_state unrescuing.
Apr 12 03:36:54 user nova-compute[72081]: DEBUG nova.compute.manager [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] Acquiring lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:54 user nova-compute[72081]: DEBUG nova.compute.manager [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] No waiting events found dispatching network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:54 user nova-compute[72081]: WARNING nova.compute.manager [req-f92ca8b3-9ea1-48af-bd0b-aba298006ff4 req-17cc4d8b-7a1b-44bb-8d85-e94365c1e566 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received unexpected event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 for instance with vm_state rescued and task_state unrescuing.
Apr 12 03:36:56 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Removed pending event for bcd267d2-5b2c-4ed2-b634-0210fae94dfd due to event {{(pid=72081) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 12 03:36:56 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270616.3438022, bcd267d2-5b2c-4ed2-b634-0210fae94dfd => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:36:56 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] VM Resumed (Lifecycle Event)
Apr 12 03:36:56 user nova-compute[72081]: DEBUG nova.compute.manager [None req-c84c9787-e70c-4934-b03d-0e2c4ef93822 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:36:56 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:36:56 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:36:56 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] During sync_power_state the instance has a pending task (unrescuing). Skip.
Apr 12 03:36:56 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270616.3592143, bcd267d2-5b2c-4ed2-b634-0210fae94dfd => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:36:56 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] VM Started (Lifecycle Event)
Apr 12 03:36:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:56 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:36:56 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Synchronizing instance power state after lifecycle event "Started"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:36:56 user nova-compute[72081]: DEBUG nova.compute.manager [req-1bf55aa0-04ba-4f9e-87f5-361f518aaf87 req-43c2a83b-d5cb-456d-8e94-bbaf2c86fb84 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1bf55aa0-04ba-4f9e-87f5-361f518aaf87 req-43c2a83b-d5cb-456d-8e94-bbaf2c86fb84 service nova] Acquiring lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1bf55aa0-04ba-4f9e-87f5-361f518aaf87 req-43c2a83b-d5cb-456d-8e94-bbaf2c86fb84 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1bf55aa0-04ba-4f9e-87f5-361f518aaf87 req-43c2a83b-d5cb-456d-8e94-bbaf2c86fb84 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:56 user nova-compute[72081]: DEBUG nova.compute.manager [req-1bf55aa0-04ba-4f9e-87f5-361f518aaf87 req-43c2a83b-d5cb-456d-8e94-bbaf2c86fb84 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] No waiting events found dispatching network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:56 user nova-compute[72081]: WARNING nova.compute.manager [req-1bf55aa0-04ba-4f9e-87f5-361f518aaf87 req-43c2a83b-d5cb-456d-8e94-bbaf2c86fb84 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received unexpected event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 for instance with vm_state active and task_state None.
Apr 12 03:36:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "49d33636-1dbb-4863-b0f3-9f1578e23e67" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "49d33636-1dbb-4863-b0f3-9f1578e23e67" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:56 user nova-compute[72081]: INFO nova.compute.manager [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Terminating instance
Apr 12 03:36:56 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:36:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:57 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Instance destroyed successfully.
Apr 12 03:36:57 user nova-compute[72081]: DEBUG nova.objects.instance [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lazy-loading 'resources' on Instance uuid 49d33636-1dbb-4863-b0f3-9f1578e23e67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:35:05Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1468791143',display_name='tempest-TestVolumeBootPattern-server-1468791143',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1468791143',id=18,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:35:23Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='003688353e2f4b36831c574ebaf65173',ramdisk_id='',reservation_id='r-v7szkyzi',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeBootPattern-235043461',owner_user_name='tempest-TestVolumeBootPattern-235043461-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:35:23Z,user_data=None,user_id='b00b348933224558aad1cb12921f100f',uuid=49d33636-1dbb-4863-b0f3-9f1578e23e67,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "93c27018-4ee9-42be-81a1-2b9377a689f5", "address": "fa:16:3e:85:8d:3c", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap93c27018-4e", "ovs_interfaceid": "93c27018-4ee9-42be-81a1-2b9377a689f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converting VIF {"id": "93c27018-4ee9-42be-81a1-2b9377a689f5", "address": "fa:16:3e:85:8d:3c", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap93c27018-4e", "ovs_interfaceid": "93c27018-4ee9-42be-81a1-2b9377a689f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:85:8d:3c,bridge_name='br-int',has_traffic_filtering=True,id=93c27018-4ee9-42be-81a1-2b9377a689f5,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap93c27018-4e') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG os_vif [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:85:8d:3c,bridge_name='br-int',has_traffic_filtering=True,id=93c27018-4ee9-42be-81a1-2b9377a689f5,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap93c27018-4e') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap93c27018-4e, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:36:57 user nova-compute[72081]: INFO os_vif [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:85:8d:3c,bridge_name='br-int',has_traffic_filtering=True,id=93c27018-4ee9-42be-81a1-2b9377a689f5,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap93c27018-4e')
Apr 12 03:36:57 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'QptXErmF', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:36:57 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:36:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1a59a815-e5c3-451d-ad6f-77a96460b25e]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:36:57 user nova-compute[72081]: No targets found.
Apr 12 03:36:57 user nova-compute[72081]: iSNS:
Apr 12 03:36:57 user nova-compute[72081]: No targets found.
Apr 12 03:36:57 user nova-compute[72081]: STATIC:
Apr 12 03:36:57 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:36:57 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:36:57 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:36:57 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f
Apr 12 03:36:57 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:36:57 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:36:57 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55
Apr 12 03:36:57 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:36:57 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:36:57 user nova-compute[72081]: FIRMWARE:
Apr 12 03:36:57 user nova-compute[72081]: No targets found.
Apr 12 03:36:57 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:36:57 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:36:57 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[75e57783-423d-4060-83c6-d50545460256]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7edc0df3-1d3f-4b89-bccd-0cddd09c051d]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:57 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:57 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:36:57 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:36:57 user nova-compute[72081]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f (non-flash)
Apr 12 03:36:57 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:36:57 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fb6fabd90>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f'): ({'sde'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Removing single pathed devices sde {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[197f44ad-48f4-4d3b-bf06-eab6ef2ed7e8]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.033s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[24e6a0f4-f5fa-4108-b73c-bb4c4c833dc1]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Flushing IO for device /dev/sde {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[93f542be-dac3-49fd-beb6-1991e7795c7b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.042s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[479dcf53-e0c7-49fe-9ab2-67d5f1d9b91f]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] SCSI volumes sde have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bee1d395-8783-40da-aebb-e4f473d4a17d]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f -p 172.16.0.220:3260 --logout" returned: 0 in 0.059s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3c731b47-cabd-4841-9178-bcf6bda63903]: (4, ('Logging out of session [sid: 17, target: iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f, portal: 172.16.0.220,3260]\nLogout of [sid: 17, target: iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 17, target: iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f, portal: 172.16.0.220,3260]
Apr 12 03:36:57 user nova-compute[72081]: Logout of [sid: 17, target: iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f, portal: 172.16.0.220,3260] successful.
Apr 12 03:36:57 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4bd5227e-f472-471d-8e4f-7088f977745f -p 172.16.0.220:3260 --op delete" returned: 0 in 0.029s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2a40c250-c5d7-40c5-928b-9e31aa075368]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.346s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] <== disconnect_volume: return (348ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:36:57 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Deleting instance files /opt/stack/data/nova/instances/49d33636-1dbb-4863-b0f3-9f1578e23e67_del
Apr 12 03:36:57 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Deletion of /opt/stack/data/nova/instances/49d33636-1dbb-4863-b0f3-9f1578e23e67_del complete
Apr 12 03:36:57 user nova-compute[72081]: INFO nova.compute.manager [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Took 1.22 seconds to destroy the instance on the hypervisor.
Apr 12 03:36:57 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:36:58 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Took 0.76 seconds to deallocate network for instance.
Apr 12 03:36:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Received event network-vif-unplugged-93c27018-4ee9-42be-81a1-2b9377a689f5 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] Acquiring lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] Lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] Lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] No waiting events found dispatching network-vif-unplugged-93c27018-4ee9-42be-81a1-2b9377a689f5 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Received event network-vif-unplugged-93c27018-4ee9-42be-81a1-2b9377a689f5 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Received event network-vif-plugged-93c27018-4ee9-42be-81a1-2b9377a689f5 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] Acquiring lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] Lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] Lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] No waiting events found dispatching network-vif-plugged-93c27018-4ee9-42be-81a1-2b9377a689f5 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:58 user nova-compute[72081]: WARNING nova.compute.manager [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Received unexpected event network-vif-plugged-93c27018-4ee9-42be-81a1-2b9377a689f5 for instance with vm_state active and task_state deleting.
Apr 12 03:36:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Received event network-vif-plugged-93c27018-4ee9-42be-81a1-2b9377a689f5 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] Acquiring lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] Lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] Lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] No waiting events found dispatching network-vif-plugged-93c27018-4ee9-42be-81a1-2b9377a689f5 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:58 user nova-compute[72081]: WARNING nova.compute.manager [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Received unexpected event network-vif-plugged-93c27018-4ee9-42be-81a1-2b9377a689f5 for instance with vm_state active and task_state deleting.
Apr 12 03:36:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Received event network-vif-plugged-93c27018-4ee9-42be-81a1-2b9377a689f5 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] Acquiring lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] Lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] Lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] No waiting events found dispatching network-vif-plugged-93c27018-4ee9-42be-81a1-2b9377a689f5 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:58 user nova-compute[72081]: WARNING nova.compute.manager [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Received unexpected event network-vif-plugged-93c27018-4ee9-42be-81a1-2b9377a689f5 for instance with vm_state active and task_state deleting.
Apr 12 03:36:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Received event network-vif-unplugged-93c27018-4ee9-42be-81a1-2b9377a689f5 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] Acquiring lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] Lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] Lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] No waiting events found dispatching network-vif-unplugged-93c27018-4ee9-42be-81a1-2b9377a689f5 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Received event network-vif-unplugged-93c27018-4ee9-42be-81a1-2b9377a689f5 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Received event network-vif-plugged-93c27018-4ee9-42be-81a1-2b9377a689f5 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] Acquiring lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] Lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] Lock "49d33636-1dbb-4863-b0f3-9f1578e23e67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] No waiting events found dispatching network-vif-plugged-93c27018-4ee9-42be-81a1-2b9377a689f5 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:36:58 user nova-compute[72081]: WARNING nova.compute.manager [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Received unexpected event network-vif-plugged-93c27018-4ee9-42be-81a1-2b9377a689f5 for instance with vm_state active and task_state deleting.
Apr 12 03:36:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-6803e734-9cfa-4515-b654-a4ce48511795 req-d2a2491d-0de5-489d-948d-be9e0c6383ae service nova] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Received event network-vif-deleted-93c27018-4ee9-42be-81a1-2b9377a689f5 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:36:59 user nova-compute[72081]: INFO nova.compute.manager [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Took 0.79 seconds to detach 1 volumes for instance.
Apr 12 03:36:59 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Deleting volume: 4bd5227e-f472-471d-8e4f-7088f977745f {{(pid=72081) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3202}}
Apr 12 03:36:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:36:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:36:59 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:36:59 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:36:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.317s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:36:59 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Deleted allocations for instance 49d33636-1dbb-4863-b0f3-9f1578e23e67
Apr 12 03:36:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1f154fc7-6e79-485f-a212-ec6c13c9a687 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "49d33636-1dbb-4863-b0f3-9f1578e23e67" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.356s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:00 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:37:01 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:37:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:03 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:37:03 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:37:03 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:37:03 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:37:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:37:03 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:37:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:03 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:37:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:03 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'flavor' on Instance uuid 48977eb2-ce43-4577-a1cb-340e35703dbd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:37:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.057s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:03 user nova-compute[72081]: INFO nova.compute.manager [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Attaching volume 0ca0118b-5d36-40b7-bb39-806756144b3a to /dev/vdb
Apr 12 03:37:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG os_brick.utils [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6cf36f18-28dd-48a2-bab1-67b5eb6b2a39]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6380d324-d280-4dfe-a882-15ef7c107f93 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Acquiring lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6380d324-d280-4dfe-a882-15ef7c107f93 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6380d324-d280-4dfe-a882-15ef7c107f93 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Acquiring lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6380d324-d280-4dfe-a882-15ef7c107f93 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6380d324-d280-4dfe-a882-15ef7c107f93 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4aa0873a-16ca-47e8-b981-9d53e994eb13]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:04 user nova-compute[72081]: INFO nova.compute.manager [None req-6380d324-d280-4dfe-a882-15ef7c107f93 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Terminating instance
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG nova.compute.manager [None req-6380d324-d280-4dfe-a882-15ef7c107f93 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[101dec07-9ee1-4b05-8aba-334a7ef3e490]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2758742a-f12c-423b-9a64-598320d22c9c]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:37:04 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[45d22693-b3a3-4b68-8aa5-328fbb70072b]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG os_brick.utils [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] <== get_connector_properties: return (188ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Updating existing volume attachment record: 695cd2ae-d0fa-4be6-b047-a23c8a5f5ca7 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json" returned: 0 in 0.161s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-7e3cda29-bd22-405b-87f1-6b94c1b803ae req-4d284b48-b2ac-40bf-bce6-f749685280f7 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Received event network-vif-unplugged-b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7e3cda29-bd22-405b-87f1-6b94c1b803ae req-4d284b48-b2ac-40bf-bce6-f749685280f7 service nova] Acquiring lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7e3cda29-bd22-405b-87f1-6b94c1b803ae req-4d284b48-b2ac-40bf-bce6-f749685280f7 service nova] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7e3cda29-bd22-405b-87f1-6b94c1b803ae req-4d284b48-b2ac-40bf-bce6-f749685280f7 service nova] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-7e3cda29-bd22-405b-87f1-6b94c1b803ae req-4d284b48-b2ac-40bf-bce6-f749685280f7 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] No waiting events found dispatching network-vif-unplugged-b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-7e3cda29-bd22-405b-87f1-6b94c1b803ae req-4d284b48-b2ac-40bf-bce6-f749685280f7 service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Received event network-vif-unplugged-b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:04 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Instance destroyed successfully.
Apr 12 03:37:04 user nova-compute[72081]: DEBUG nova.objects.instance [None req-6380d324-d280-4dfe-a882-15ef7c107f93 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lazy-loading 'resources' on Instance uuid 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-6380d324-d280-4dfe-a882-15ef7c107f93 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-12T03:34:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-934756275',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-934756275',id=15,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGqZ2JOd0oMveikCRo3egpYXw6F7M9CSnvg8uuSzQFHFmYKOQ1+3ysT+GbPl8KtibugwjCH/8obQl0aNd57dyywpuoimEfD8++mntdrmImk7chBwlJ4Kqnvt9BRQOwzT8Q==',key_name='tempest-keypair-52848813',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:34:18Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='a327d22640aa4a7caeb35fb0ed2dccc8',ramdisk_id='',reservation_id='r-autn96ne',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-1144278074',owner_user_name='tempest-TaggedAttachmentsTest-1144278074-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:34:19Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3a7c2b6c06a745de95fae59f8b02c06d',uuid=37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "address": "fa:16:3e:46:28:01", "network": {"id": "8d284b3a-85a8-45a3-845f-271f3fe80540", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-354475413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.20", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a327d22640aa4a7caeb35fb0ed2dccc8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb0b22dc0-c2", "ovs_interfaceid": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-6380d324-d280-4dfe-a882-15ef7c107f93 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Converting VIF {"id": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "address": "fa:16:3e:46:28:01", "network": {"id": "8d284b3a-85a8-45a3-845f-271f3fe80540", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-354475413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.20", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a327d22640aa4a7caeb35fb0ed2dccc8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb0b22dc0-c2", "ovs_interfaceid": "b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-6380d324-d280-4dfe-a882-15ef7c107f93 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:46:28:01,bridge_name='br-int',has_traffic_filtering=True,id=b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958,network=Network(8d284b3a-85a8-45a3-845f-271f3fe80540),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb0b22dc0-c2') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG os_vif [None req-6380d324-d280-4dfe-a882-15ef7c107f93 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:46:28:01,bridge_name='br-int',has_traffic_filtering=True,id=b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958,network=Network(8d284b3a-85a8-45a3-845f-271f3fe80540),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb0b22dc0-c2') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapb0b22dc0-c2, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:37:04 user nova-compute[72081]: INFO os_vif [None req-6380d324-d280-4dfe-a882-15ef7c107f93 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:46:28:01,bridge_name='br-int',has_traffic_filtering=True,id=b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958,network=Network(8d284b3a-85a8-45a3-845f-271f3fe80540),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb0b22dc0-c2')
Apr 12 03:37:04 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-6380d324-d280-4dfe-a882-15ef7c107f93 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Deleting instance files /opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63_del
Apr 12 03:37:04 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-6380d324-d280-4dfe-a882-15ef7c107f93 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Deletion of /opt/stack/data/nova/instances/37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63_del complete
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:04 user nova-compute[72081]: INFO nova.compute.manager [None req-6380d324-d280-4dfe-a882-15ef7c107f93 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Took 0.90 seconds to destroy the instance on the hypervisor.
Apr 12 03:37:04 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-6380d324-d280-4dfe-a882-15ef7c107f93 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:37:04 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:37:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk --force-share --output=json" returned: 0 in 0.166s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk --force-share --output=json" returned: 0 in 0.159s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:37:05 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Error from libvirt while getting description of instance-0000000f: [Error Code 42] Domain not found: no domain with matching uuid '37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63' (instance-0000000f): libvirt.libvirtError: Domain not found: no domain with matching uuid '37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63' (instance-0000000f)
Apr 12 03:37:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:05 user nova-compute[72081]: DEBUG nova.compute.manager [req-a51edb7f-9713-409f-8ed1-6c4a1ff3ace4 req-38cc13f3-1def-4c26-a8b7-0477a4f88339 service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Received event network-changed-892ff129-c9ca-4295-be61-993b81b855fb {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:37:05 user nova-compute[72081]: DEBUG nova.compute.manager [req-a51edb7f-9713-409f-8ed1-6c4a1ff3ace4 req-38cc13f3-1def-4c26-a8b7-0477a4f88339 service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Refreshing instance network info cache due to event network-changed-892ff129-c9ca-4295-be61-993b81b855fb. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:37:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a51edb7f-9713-409f-8ed1-6c4a1ff3ace4 req-38cc13f3-1def-4c26-a8b7-0477a4f88339 service nova] Acquiring lock "refresh_cache-a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:37:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a51edb7f-9713-409f-8ed1-6c4a1ff3ace4 req-38cc13f3-1def-4c26-a8b7-0477a4f88339 service nova] Acquired lock "refresh_cache-a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:37:05 user nova-compute[72081]: DEBUG nova.network.neutron [req-a51edb7f-9713-409f-8ed1-6c4a1ff3ace4 req-38cc13f3-1def-4c26-a8b7-0477a4f88339 service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Refreshing network info cache for port 892ff129-c9ca-4295-be61-993b81b855fb {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:37:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:06 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:37:06 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:37:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=7806MB free_disk=26.06472396850586GB free_vcpus=4 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 56834002-7593-4592-a86d-23bb90d00be0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 2d0904c3-74e0-47bb-a832-758a7035b2f5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 72ba1218-065d-40c4-8503-7c4155e47503 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance bcd267d2-5b2c-4ed2-b634-0210fae94dfd actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 48977eb2-ce43-4577-a1cb-340e35703dbd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 23367b24-0a3b-454d-901d-8a3e5396333a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 8 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=1536MB phys_disk=40GB used_disk=7GB total_vcpus=12 used_vcpus=8 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:37:06 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Took 1.41 seconds to deallocate network for instance.
Apr 12 03:37:06 user nova-compute[72081]: DEBUG nova.compute.manager [req-6b2289ff-5f93-48d1-afef-8104e7b91e72 req-e948400e-de71-49dd-aded-601e329316de service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Received event network-vif-plugged-b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6b2289ff-5f93-48d1-afef-8104e7b91e72 req-e948400e-de71-49dd-aded-601e329316de service nova] Acquiring lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6b2289ff-5f93-48d1-afef-8104e7b91e72 req-e948400e-de71-49dd-aded-601e329316de service nova] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6b2289ff-5f93-48d1-afef-8104e7b91e72 req-e948400e-de71-49dd-aded-601e329316de service nova] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG nova.compute.manager [req-6b2289ff-5f93-48d1-afef-8104e7b91e72 req-e948400e-de71-49dd-aded-601e329316de service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] No waiting events found dispatching network-vif-plugged-b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:37:06 user nova-compute[72081]: WARNING nova.compute.manager [req-6b2289ff-5f93-48d1-afef-8104e7b91e72 req-e948400e-de71-49dd-aded-601e329316de service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Received unexpected event network-vif-plugged-b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958 for instance with vm_state active and task_state deleting.
Apr 12 03:37:06 user nova-compute[72081]: DEBUG nova.compute.manager [req-6b2289ff-5f93-48d1-afef-8104e7b91e72 req-e948400e-de71-49dd-aded-601e329316de service nova] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Received event network-vif-deleted-b0b22dc0-c2cf-4aed-b5a2-a9bfddcd1958 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6380d324-d280-4dfe-a882-15ef7c107f93 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.435s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6380d324-d280-4dfe-a882-15ef7c107f93 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.204s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG nova.network.neutron [req-a51edb7f-9713-409f-8ed1-6c4a1ff3ace4 req-38cc13f3-1def-4c26-a8b7-0477a4f88339 service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Updated VIF entry in instance network info cache for port 892ff129-c9ca-4295-be61-993b81b855fb. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG nova.network.neutron [req-a51edb7f-9713-409f-8ed1-6c4a1ff3ace4 req-38cc13f3-1def-4c26-a8b7-0477a4f88339 service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Updating instance_info_cache with network_info: [{"id": "892ff129-c9ca-4295-be61-993b81b855fb", "address": "fa:16:3e:f0:b1:a0", "network": {"id": "263f9508-8639-4289-9210-f9807acf6766", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-337713645-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.195", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f4cce206e2b94b03ade2d310373c5ca2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap892ff129-c9", "ovs_interfaceid": "892ff129-c9ca-4295-be61-993b81b855fb", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a51edb7f-9713-409f-8ed1-6c4a1ff3ace4 req-38cc13f3-1def-4c26-a8b7-0477a4f88339 service nova] Releasing lock "refresh_cache-a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-6380d324-d280-4dfe-a882-15ef7c107f93 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-6380d324-d280-4dfe-a882-15ef7c107f93 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:37:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6380d324-d280-4dfe-a882-15ef7c107f93 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.305s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:06 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-6380d324-d280-4dfe-a882-15ef7c107f93 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Deleted allocations for instance 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63
Apr 12 03:37:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6380d324-d280-4dfe-a882-15ef7c107f93 tempest-TaggedAttachmentsTest-1144278074 tempest-TaggedAttachmentsTest-1144278074-project-member] Lock "37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.025s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquiring lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG nova.objects.instance [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lazy-loading 'flavor' on Instance uuid a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.055s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquiring lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:07 user nova-compute[72081]: INFO nova.compute.manager [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Attaching volume c59a4756-2e14-4cf2-8c9d-e6da704aaa4b to /dev/vdb
Apr 12 03:37:07 user nova-compute[72081]: DEBUG os_brick.utils [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[88e9404d-c470-4429-b418-c529d2eb8cc8]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[af7a733a-8158-4859-944b-4944cbbc3b9b]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[39b2054c-810c-439c-9903-674627e70a82]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2ca615b9-9d81-4a2c-8ca7-08d7c5b0e715]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:37:07 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:37:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a033de93-caf4-4575-8e6c-e11c119cb0c5]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG os_brick.utils [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] <== get_connector_properties: return (112ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Updating existing volume attachment record: 1d8bfb80-c8b9-49c1-9fc8-a051ca6812a8 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 03:37:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Updating instance_info_cache with network_info: [{"id": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "address": "fa:16:3e:c0:f6:ee", "network": {"id": "b4464ad0-6022-43e7-bfe4-229e6a3a0600", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1704232678-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.188", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a027d7708e6747fe970c388d6ba66359", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcff588d4-f4", "ovs_interfaceid": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-72ba1218-065d-40c4-8503-7c4155e47503" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'aYQxDBlt', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:37:08 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:37:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:37:08 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:37:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:37:08 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e1f8d1ac-6073-427c-ab74-07d9a5632f8a]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:37:08 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e3689c9a-7f2b-4e76-ba0e-09fc61d95ede]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a] added
Apr 12 03:37:08 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[957228bd-2499-429c-846b-5f20d76f856b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9fa91c4a-aef0-4640-962a-b26417fc9bd7]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op update -n node.session.auth.username -v aYQxDBlt {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op update -n node.session.auth.username -v aYQxDBlt" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e261ac34-91fc-49f1-9e72-0ea86ed63b8f]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'aYQxDBlt'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[929f6363-3a85-4ce3-bc73-605a3c7ef3c9]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[53fc1275-1ea7-4d11-8475-cc331a051bd8]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:37:08 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:37:08 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:37:08 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:37:08 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --login" returned: 0 in 0.058s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[00568ed7-45da-4c43-9f33-90565632cf8a]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a, portal: 172.16.0.220,3260]
Apr 12 03:37:08 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a, portal: 172.16.0.220,3260] successful.
Apr 12 03:37:08 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[84c39111-5f25-4151-87c9-19363131620a]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[199efa65-c202-43ae-ad2e-6cfae0ce0843]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:37:08 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:37:08 user nova-compute[72081]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)
Apr 12 03:37:08 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:37:08 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:37:08 user nova-compute[72081]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)
Apr 12 03:37:08 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.265s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] HCTL ('35', '-', '-', 0) found on session 22 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6a412043-0b0d-431f-8a65-7a742bbc32bb]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Searching for a device in session 22 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:37:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'aYQxDBlt', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:37:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.295s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:37:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] <== connect_volume: return (1297ms) {'type': 'block', 'scsi_wwn': '26539336364643435', 'path': '/dev/sdd'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:37:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26539336364643435', 'path': '/dev/sdd'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:37:09 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'flavor' on Instance uuid 48977eb2-ce43-4577-a1cb-340e35703dbd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:37:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] attach device xml: <disk type="block" device="disk">
Apr 12 03:37:09 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:37:09 user nova-compute[72081]:   <source dev="/dev/sdd"/>
Apr 12 03:37:09 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:37:09 user nova-compute[72081]:   <serial>0ca0118b-5d36-40b7-bb39-806756144b3a</serial>
Apr 12 03:37:09 user nova-compute[72081]: </disk>
Apr 12 03:37:09 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 03:37:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:37:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:37:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] No VIF found with MAC fa:16:3e:b3:c5:ca, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:37:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9fabea37-f42e-4657-928a-98f2c0b37493 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.479s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'iYueoLfD', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:37:11 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:37:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:37:11 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:37:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:37:11 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[60d3b2a7-be06-4e34-8b90-cbfdc1352cec]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:37:11 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5b3dcdb3-f92f-4772-8e44-6d8ed62a92bd]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b] added
Apr 12 03:37:11 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[28f28d29-d13c-4469-8641-179f1eccfc0a]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[97f58d04-7202-42d5-b648-f0a14209d047]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v iYueoLfD {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v iYueoLfD" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[eb7bb337-2ea6-4260-8c38-e58fdd43fb0f]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'iYueoLfD'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[faa2ce0e-e484-481b-9b0a-0dae28044ba9]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[36c52db7-7345-4e61-b8b6-ac5a4aabef81]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:37:11 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:37:11 user nova-compute[72081]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)
Apr 12 03:37:11 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:37:11 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:37:11 user nova-compute[72081]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)
Apr 12 03:37:11 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b -p 172.16.0.220:3260 --login" returned: 0 in 0.069s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d4d00a5c-595f-44ef-be9a-4f4cc3cda7bf]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b, portal: 172.16.0.220,3260]
Apr 12 03:37:11 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b, portal: 172.16.0.220,3260] successful.
Apr 12 03:37:11 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e2bb9ef4-cb44-4371-b8b7-8adc6814e6de]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[14b78bd1-bdcb-452d-bb20-575884a77a35]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:37:11 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:37:11 user nova-compute[72081]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)
Apr 12 03:37:11 user nova-compute[72081]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b (non-flash)
Apr 12 03:37:11 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:37:11 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:37:11 user nova-compute[72081]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)
Apr 12 03:37:11 user nova-compute[72081]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b (non-flash)
Apr 12 03:37:11 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.294s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] HCTL ('36', '-', '-', 0) found on session 23 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:37:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:12 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[58429f7c-f64f-4ffe-8a22-ab210242fa65]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:12 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Searching for a device in session 23 and hctl ['36', '*', '*', 0] yield: sde {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:37:12 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'iYueoLfD', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:37:12 user nova-compute[72081]: INFO nova.compute.manager [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Rebuilding instance
Apr 12 03:37:12 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'trusted_certs' on Instance uuid 48977eb2-ce43-4577-a1cb-340e35703dbd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:37:12 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:37:12 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'pci_requests' on Instance uuid 48977eb2-ce43-4577-a1cb-340e35703dbd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:37:12 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'pci_devices' on Instance uuid 48977eb2-ce43-4577-a1cb-340e35703dbd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:37:12 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'resources' on Instance uuid 48977eb2-ce43-4577-a1cb-340e35703dbd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:37:12 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'migration_context' on Instance uuid 48977eb2-ce43-4577-a1cb-340e35703dbd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:37:12 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Trying to apply a migration context that does not seem to be set for this instance {{(pid=72081) apply_migration_context /opt/stack/nova/nova/objects/instance.py:1032}}
Apr 12 03:37:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:12 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681270617.3725772, 49d33636-1dbb-4863-b0f3-9f1578e23e67 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:37:12 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] VM Stopped (Lifecycle Event)
Apr 12 03:37:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:12 user nova-compute[72081]: DEBUG nova.compute.manager [None req-558b6410-d05d-49ed-80c2-12819f4a537f None None] [instance: 49d33636-1dbb-4863-b0f3-9f1578e23e67] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:37:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:12 user nova-compute[72081]: DEBUG nova.compute.manager [req-82d8e0bc-5d51-4fc1-b825-3b6fcb41fa42 req-a2d77430-38c9-4a9e-ae0a-fb0c04936da8 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received event network-vif-unplugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:37:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-82d8e0bc-5d51-4fc1-b825-3b6fcb41fa42 req-a2d77430-38c9-4a9e-ae0a-fb0c04936da8 service nova] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-82d8e0bc-5d51-4fc1-b825-3b6fcb41fa42 req-a2d77430-38c9-4a9e-ae0a-fb0c04936da8 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-82d8e0bc-5d51-4fc1-b825-3b6fcb41fa42 req-a2d77430-38c9-4a9e-ae0a-fb0c04936da8 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:12 user nova-compute[72081]: DEBUG nova.compute.manager [req-82d8e0bc-5d51-4fc1-b825-3b6fcb41fa42 req-a2d77430-38c9-4a9e-ae0a-fb0c04936da8 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] No waiting events found dispatching network-vif-unplugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:37:12 user nova-compute[72081]: WARNING nova.compute.manager [req-82d8e0bc-5d51-4fc1-b825-3b6fcb41fa42 req-a2d77430-38c9-4a9e-ae0a-fb0c04936da8 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received unexpected event network-vif-unplugged-18f9f68c-eadd-4582-8f16-b88a13496852 for instance with vm_state active and task_state rebuilding.
Apr 12 03:37:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.324s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] <== connect_volume: return (1325ms) {'type': 'block', 'scsi_wwn': '23435343461313866', 'path': '/dev/sde'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23435343461313866', 'path': '/dev/sde'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG nova.objects.instance [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lazy-loading 'flavor' on Instance uuid a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] attach device xml: <disk type="block" device="disk">
Apr 12 03:37:13 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:37:13 user nova-compute[72081]:   <source dev="/dev/sde"/>
Apr 12 03:37:13 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:37:13 user nova-compute[72081]:   <serial>c59a4756-2e14-4cf2-8c9d-e6da704aaa4b</serial>
Apr 12 03:37:13 user nova-compute[72081]: </disk>
Apr 12 03:37:13 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 03:37:13 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Instance destroyed successfully.
Apr 12 03:37:13 user nova-compute[72081]: INFO nova.compute.manager [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Detaching volume 0ca0118b-5d36-40b7-bb39-806756144b3a
Apr 12 03:37:13 user nova-compute[72081]: INFO nova.virt.block_device [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Attempting to driver detach volume 0ca0118b-5d36-40b7-bb39-806756144b3a from mountpoint /dev/vdb
Apr 12 03:37:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Attempting to detach device vdb from instance 48977eb2-ce43-4577-a1cb-340e35703dbd from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:37:13 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:37:13 user nova-compute[72081]:   <source dev="/dev/sdd"/>
Apr 12 03:37:13 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:37:13 user nova-compute[72081]:   <serial>0ca0118b-5d36-40b7-bb39-806756144b3a</serial>
Apr 12 03:37:13 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 03:37:13 user nova-compute[72081]: </disk>
Apr 12 03:37:13 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:37:13 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Successfully detached device vdb from instance 48977eb2-ce43-4577-a1cb-340e35703dbd from the persistent domain config.
Apr 12 03:37:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'aYQxDBlt', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:37:13 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Apr 12 03:37:13 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.004s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[24c979d8-8e57-431e-8e66-edaf3b0d3407]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:37:13 user nova-compute[72081]: No targets found.
Apr 12 03:37:13 user nova-compute[72081]: iSNS:
Apr 12 03:37:13 user nova-compute[72081]: No targets found.
Apr 12 03:37:13 user nova-compute[72081]: STATIC:
Apr 12 03:37:13 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:37:13 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:37:13 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:37:13 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a
Apr 12 03:37:13 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:37:13 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:37:13 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b
Apr 12 03:37:13 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:37:13 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:37:13 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55
Apr 12 03:37:13 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:37:13 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:37:13 user nova-compute[72081]: FIRMWARE:
Apr 12 03:37:13 user nova-compute[72081]: No targets found.
Apr 12 03:37:13 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:37:13 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:37:13 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5fdf3a78-df05-4b0a-a36a-2f7581f40736]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1fdd4f6b-8c33-4c7e-a42f-41d7e1e41a46]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:37:13 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:37:13 user nova-compute[72081]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)
Apr 12 03:37:13 user nova-compute[72081]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b (non-flash)
Apr 12 03:37:13 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:37:13 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:37:13 user nova-compute[72081]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)
Apr 12 03:37:13 user nova-compute[72081]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b (non-flash)
Apr 12 03:37:13 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fb420f130>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a'): ({'sdd'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Removing single pathed devices sdd {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9bdb0bfb-bc44-436d-8ed4-c9a46cbcbbbe]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ad4bc1d8-6e79-4eba-8be9-932aca328376]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Flushing IO for device /dev/sdd {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.013s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0e310def-e15d-4f2b-aa50-688e8cdbd34b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] No VIF found with MAC fa:16:3e:f0:b1:a0, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.044s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[291215a2-a998-45fd-b4d6-41ce9e76fbf7]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] SCSI volumes sdd have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[332a4377-63a8-4f7e-9574-46d76590f64c]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-43c29eb3-16e6-4d38-a5d3-2378314892e4 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.231s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --logout" returned: 0 in 0.081s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c9ce3db8-f022-4bdf-bd00-d32888822847]: (4, ('Logging out of session [sid: 22, target: iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a, portal: 172.16.0.220,3260]\nLogout of [sid: 22, target: iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 22, target: iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a, portal: 172.16.0.220,3260]
Apr 12 03:37:13 user nova-compute[72081]: Logout of [sid: 22, target: iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a, portal: 172.16.0.220,3260] successful.
Apr 12 03:37:13 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op delete" returned: 0 in 0.031s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b45ecd0f-9145-49e3-9204-93d51bcd2a53]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.368s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] <== disconnect_volume: return (373ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquiring lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:14 user nova-compute[72081]: INFO nova.compute.manager [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Detaching volume c59a4756-2e14-4cf2-8c9d-e6da704aaa4b
Apr 12 03:37:14 user nova-compute[72081]: INFO nova.virt.block_device [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Attempting to driver detach volume c59a4756-2e14-4cf2-8c9d-e6da704aaa4b from mountpoint /dev/vdb
Apr 12 03:37:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Attempting to detach device vdb from instance a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6 from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:37:14 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:37:14 user nova-compute[72081]:   <source dev="/dev/sde"/>
Apr 12 03:37:14 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:37:14 user nova-compute[72081]:   <serial>c59a4756-2e14-4cf2-8c9d-e6da704aaa4b</serial>
Apr 12 03:37:14 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 03:37:14 user nova-compute[72081]: </disk>
Apr 12 03:37:14 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:37:14 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Successfully detached device vdb from instance a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6 from the persistent domain config.
Apr 12 03:37:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6 from the live domain config. {{(pid=72081) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:37:14 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:37:14 user nova-compute[72081]:   <source dev="/dev/sde"/>
Apr 12 03:37:14 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:37:14 user nova-compute[72081]:   <serial>c59a4756-2e14-4cf2-8c9d-e6da704aaa4b</serial>
Apr 12 03:37:14 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 03:37:14 user nova-compute[72081]: </disk>
Apr 12 03:37:14 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:37:14 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Instance destroyed successfully.
Apr 12 03:37:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:33:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-2050314981',display_name='tempest-ServerActionsTestJSON-server-1847743592',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-2050314981',id=14,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKmrI9wy1CUIwhlKuXvzk0mD7Pt5nZFvI1CsqS0do6Lk76JQOvGl5lvx9rkz7UAM9WGocGXyWSOUOuKTtYZRD2HqKUbTN4VPOoliE1yuu+s0lx7vuZu45Mj7wDb2S8Ksyw==',key_name='tempest-keypair-364276420',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:33:27Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={rebuild='server'},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='815b65aa5ad9442db0a087c646f96bbd',ramdisk_id='',reservation_id='r-usz98205',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestJSON-1898059859',owner_user_name='tempest-ServerActionsTestJSON-1898059859-project-member'},tags=<?>,task_state='rebuilding',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:37:12Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a7f422f1a05a4a92ba387e424850b360',uuid=48977eb2-ce43-4577-a1cb-340e35703dbd,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converting VIF {"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG os_vif [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap18f9f68c-ea, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG nova.compute.manager [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG nova.compute.manager [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] No waiting events found dispatching network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:37:14 user nova-compute[72081]: WARNING nova.compute.manager [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received unexpected event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 for instance with vm_state active and task_state rebuilding.
Apr 12 03:37:14 user nova-compute[72081]: DEBUG nova.compute.manager [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG nova.compute.manager [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] No waiting events found dispatching network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:37:14 user nova-compute[72081]: WARNING nova.compute.manager [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received unexpected event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 for instance with vm_state active and task_state rebuilding.
Apr 12 03:37:14 user nova-compute[72081]: DEBUG nova.compute.manager [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG nova.compute.manager [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] No waiting events found dispatching network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:37:14 user nova-compute[72081]: WARNING nova.compute.manager [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received unexpected event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 for instance with vm_state active and task_state rebuilding.
Apr 12 03:37:14 user nova-compute[72081]: DEBUG nova.compute.manager [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received event network-vif-unplugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG nova.compute.manager [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] No waiting events found dispatching network-vif-unplugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:37:14 user nova-compute[72081]: WARNING nova.compute.manager [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received unexpected event network-vif-unplugged-18f9f68c-eadd-4582-8f16-b88a13496852 for instance with vm_state active and task_state rebuilding.
Apr 12 03:37:14 user nova-compute[72081]: DEBUG nova.compute.manager [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG nova.compute.manager [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] No waiting events found dispatching network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:37:14 user nova-compute[72081]: WARNING nova.compute.manager [req-3ab09148-d743-40d2-8364-dd098bd6e5a1 req-2ca08e21-1b3b-4af6-9c66-41e45a5e7bb4 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received unexpected event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 for instance with vm_state active and task_state rebuilding.
Apr 12 03:37:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:37:14 user nova-compute[72081]: INFO os_vif [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea')
Apr 12 03:37:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'aYQxDBlt', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:37:14 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Apr 12 03:37:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb86e88f-6d0c-43a4-90c8-3ab8df70ebd4]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:37:14 user nova-compute[72081]: No targets found.
Apr 12 03:37:14 user nova-compute[72081]: iSNS:
Apr 12 03:37:14 user nova-compute[72081]: No targets found.
Apr 12 03:37:14 user nova-compute[72081]: STATIC:
Apr 12 03:37:14 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:37:14 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:37:14 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:37:14 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b
Apr 12 03:37:14 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:37:14 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:37:14 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55
Apr 12 03:37:14 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:37:14 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:37:14 user nova-compute[72081]: FIRMWARE:
Apr 12 03:37:14 user nova-compute[72081]: No targets found.
Apr 12 03:37:14 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:37:14 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:37:14 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0ee78581-5604-4389-83c9-e803d8fb0340]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4247a5a6-9a82-4541-b0ed-63e14915d782]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:37:14 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:37:14 user nova-compute[72081]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b (non-flash)
Apr 12 03:37:14 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:37:14 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:37:14 user nova-compute[72081]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b (non-flash)
Apr 12 03:37:14 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fba3b7640>, {}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Disconnecting from: [] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.069s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] <== disconnect_volume: return (70ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:37:14 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Deleting instance files /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd_del
Apr 12 03:37:14 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Deletion of /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd_del complete
Apr 12 03:37:14 user nova-compute[72081]: INFO nova.virt.block_device [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Booting with volume 0ca0118b-5d36-40b7-bb39-806756144b3a at /dev/vdb
Apr 12 03:37:14 user nova-compute[72081]: DEBUG os_brick.utils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d61d26c-8402-4c03-8bc7-1ae311bd00ab]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[535231bc-f598-48c1-9ecb-c9b43816a3bf]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2d303993-55bb-4ded-b93f-ef798ec0ba29]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9a230977-35c3-4405-9aa1-fc7b2c5b790f]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Received event <DeviceRemovedEvent: 1681270634.9435723, a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72081) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6 {{(pid=72081) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:14 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Successfully detached device vdb from instance a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6 from the live domain config.
Apr 12 03:37:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'iYueoLfD', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:37:14 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:37:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:37:14 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1f11d1ed-3cf1-4f99-8a27-9a3385092407]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG os_brick.utils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] <== get_connector_properties: return (114ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Updating existing volume attachment record: ad57b87f-f9b8-404c-b338-3aa82dde62ac {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5560f328-d46e-4c0d-ba4f-76cf87d85c6f]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:37:14 user nova-compute[72081]: No targets found.
Apr 12 03:37:14 user nova-compute[72081]: iSNS:
Apr 12 03:37:14 user nova-compute[72081]: No targets found.
Apr 12 03:37:14 user nova-compute[72081]: STATIC:
Apr 12 03:37:14 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:37:14 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:37:14 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:37:14 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b
Apr 12 03:37:14 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:37:14 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:37:14 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55
Apr 12 03:37:14 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:37:14 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:37:14 user nova-compute[72081]: FIRMWARE:
Apr 12 03:37:14 user nova-compute[72081]: No targets found.
Apr 12 03:37:14 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:37:14 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:37:14 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:37:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[061eb584-7d77-482e-9d74-d534068e87c8]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5b5120de-abb2-4561-85f1-3c00c33f1536]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:37:15 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:37:15 user nova-compute[72081]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b (non-flash)
Apr 12 03:37:15 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:37:15 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:37:15 user nova-compute[72081]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b (non-flash)
Apr 12 03:37:15 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3f9416c820>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b'): ({'sde'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Removing single pathed devices sde {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0d1d81ea-cef0-43ad-8eb9-ae92a3c3f6c9]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[21609fe5-7951-4668-9f08-50bfd423740f]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Flushing IO for device /dev/sde {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.013s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[880667a7-f9ba-485c-8613-c3deff3f14fb]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.040s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7f657f45-7257-4326-99cf-7666ca44370d]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] SCSI volumes sde have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[498d9825-c08f-473c-af97-26395057c7cc]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b -p 172.16.0.220:3260 --logout" returned: 0 in 0.048s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[10382100-751f-4bc3-b38c-0df7b4ad256c]: (4, ('Logging out of session [sid: 23, target: iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b, portal: 172.16.0.220,3260]\nLogout of [sid: 23, target: iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 23, target: iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b, portal: 172.16.0.220,3260]
Apr 12 03:37:15 user nova-compute[72081]: Logout of [sid: 23, target: iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b, portal: 172.16.0.220,3260] successful.
Apr 12 03:37:15 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c59a4756-2e14-4cf2-8c9d-e6da704aaa4b -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[cfec36c8-8bed-4623-aae3-9f1a25e68f82]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.281s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] <== disconnect_volume: return (283ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:37:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:37:16 user nova-compute[72081]: DEBUG nova.objects.instance [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lazy-loading 'flavor' on Instance uuid a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:37:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-e6caaaa3-ce7e-4f3f-a284-932c90087515 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.681s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:37:19 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Creating image(s)
Apr 12 03:37:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "/opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "/opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "/opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.131s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk 1073741824" returned: 0 in 0.047s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.183s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.127s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Checking if we can resize image /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681270624.8382888, 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:37:19 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] VM Stopped (Lifecycle Event)
Apr 12 03:37:19 user nova-compute[72081]: DEBUG nova.compute.manager [None req-a57def04-bb40-4443-9a25-3a2238cd1c94 None None] [instance: 37b9eee6-8f7b-4e74-bfc7-6e94dc8b3b63] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Cannot resize image /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Ensure instance console log exists: /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Start _get_guest_xml network_info=[{"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vdb', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'aYQxDBlt', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '48977eb2-ce43-4577-a1cb-340e35703dbd', 'attached_at': '', 'detached_at': '', 'volume_id': '0ca0118b-5d36-40b7-bb39-806756144b3a', 'serial': '0ca0118b-5d36-40b7-bb39-806756144b3a'}, 'disk_bus': 'virtio', 'delete_on_termination': False, 'boot_index': None, 'guest_format': None, 'device_type': 'disk', 'attachment_id': 'ad57b87f-f9b8-404c-b338-3aa82dde62ac', 'volume_type': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:37:19 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.: NotImplementedError
Apr 12 03:37:19 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.: NotImplementedError
Apr 12 03:37:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:37:19 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'vcpu_model' on Instance uuid 48977eb2-ce43-4577-a1cb-340e35703dbd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'aYQxDBlt', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:37:20 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Apr 12 03:37:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:37:20 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Apr 12 03:37:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:37:20 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[fee6e266-7c98-4024-96d3-73cd190e753a]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:37:20 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9235baa9-613a-4d8d-a3a7-6bf0646d1150]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a] added
Apr 12 03:37:20 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6eb3ef54-2438-4677-9b70-9810e95ea103]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6a144e14-7469-4dba-9359-04f64a80bdc0]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op update -n node.session.auth.username -v aYQxDBlt {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op update -n node.session.auth.username -v aYQxDBlt" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f06de618-05b1-45b8-956a-40c8c6385e37]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'aYQxDBlt'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c622e5d5-97db-4ae7-bca9-a95e5dcf2b3a]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6941ee29-67a3-4c49-a543-578d807e7ce9]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:37:20 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:37:20 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:37:20 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:37:20 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --login" returned: 0 in 0.067s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[345ba664-f01d-4d36-a371-d64d755d819b]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a, portal: 172.16.0.220,3260]
Apr 12 03:37:20 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a, portal: 172.16.0.220,3260] successful.
Apr 12 03:37:20 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2d858557-d3d0-4b3e-8e30-2e7e6d528a8c]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ba16ea8e-d41a-4dcd-b9a8-2d23821a50ec]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:37:20 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:37:20 user nova-compute[72081]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)
Apr 12 03:37:20 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:37:20 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:37:20 user nova-compute[72081]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)
Apr 12 03:37:20 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.300s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] HCTL ('35', '-', '-', 0) found on session 24 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[71da3b07-1581-4427-916c-8486f5e272b3]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Searching for a device in session 24 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:37:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'aYQxDBlt', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:37:21 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.333s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:37:21 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] <== connect_volume: return (1336ms) {'type': 'block', 'scsi_wwn': '26539336364643435', 'path': '/dev/sdd'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:37:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26539336364643435', 'path': '/dev/sdd'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:37:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-12T03:33:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-2050314981',display_name='tempest-ServerActionsTestJSON-server-1847743592',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-2050314981',id=14,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKmrI9wy1CUIwhlKuXvzk0mD7Pt5nZFvI1CsqS0do6Lk76JQOvGl5lvx9rkz7UAM9WGocGXyWSOUOuKTtYZRD2HqKUbTN4VPOoliE1yuu+s0lx7vuZu45Mj7wDb2S8Ksyw==',key_name='tempest-keypair-364276420',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:33:27Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={rebuild='server'},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='815b65aa5ad9442db0a087c646f96bbd',ramdisk_id='',reservation_id='r-usz98205',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestJSON-1898059859',owner_user_name='tempest-ServerActionsTestJSON-1898059859-project-member'},tags=<?>,task_state='rebuild_spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:37:15Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a7f422f1a05a4a92ba387e424850b360',uuid=48977eb2-ce43-4577-a1cb-340e35703dbd,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:37:21 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converting VIF {"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:37:21 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:37:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:37:21 user nova-compute[72081]:   <uuid>48977eb2-ce43-4577-a1cb-340e35703dbd</uuid>
Apr 12 03:37:21 user nova-compute[72081]:   <name>instance-0000000e</name>
Apr 12 03:37:21 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:37:21 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:37:21 user nova-compute[72081]:   <metadata>
Apr 12 03:37:21 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:37:21 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:37:21 user nova-compute[72081]:       <nova:name>tempest-ServerActionsTestJSON-server-1847743592</nova:name>
Apr 12 03:37:21 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:37:19</nova:creationTime>
Apr 12 03:37:21 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:37:21 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:37:21 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:37:21 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:37:21 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:37:21 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:37:21 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:37:21 user nova-compute[72081]:       <nova:owner>
Apr 12 03:37:21 user nova-compute[72081]:         <nova:user uuid="a7f422f1a05a4a92ba387e424850b360">tempest-ServerActionsTestJSON-1898059859-project-member</nova:user>
Apr 12 03:37:21 user nova-compute[72081]:         <nova:project uuid="815b65aa5ad9442db0a087c646f96bbd">tempest-ServerActionsTestJSON-1898059859</nova:project>
Apr 12 03:37:21 user nova-compute[72081]:       </nova:owner>
Apr 12 03:37:21 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:37:21 user nova-compute[72081]:       <nova:ports>
Apr 12 03:37:21 user nova-compute[72081]:         <nova:port uuid="18f9f68c-eadd-4582-8f16-b88a13496852">
Apr 12 03:37:21 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 12 03:37:21 user nova-compute[72081]:         </nova:port>
Apr 12 03:37:21 user nova-compute[72081]:       </nova:ports>
Apr 12 03:37:21 user nova-compute[72081]:     </nova:instance>
Apr 12 03:37:21 user nova-compute[72081]:   </metadata>
Apr 12 03:37:21 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:37:21 user nova-compute[72081]:     <system>
Apr 12 03:37:21 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:37:21 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:37:21 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:37:21 user nova-compute[72081]:       <entry name="serial">48977eb2-ce43-4577-a1cb-340e35703dbd</entry>
Apr 12 03:37:21 user nova-compute[72081]:       <entry name="uuid">48977eb2-ce43-4577-a1cb-340e35703dbd</entry>
Apr 12 03:37:21 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:37:21 user nova-compute[72081]:     </system>
Apr 12 03:37:21 user nova-compute[72081]:   </sysinfo>
Apr 12 03:37:21 user nova-compute[72081]:   <os>
Apr 12 03:37:21 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:37:21 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:37:21 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:37:21 user nova-compute[72081]:   </os>
Apr 12 03:37:21 user nova-compute[72081]:   <features>
Apr 12 03:37:21 user nova-compute[72081]:     <acpi/>
Apr 12 03:37:21 user nova-compute[72081]:     <apic/>
Apr 12 03:37:21 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:37:21 user nova-compute[72081]:   </features>
Apr 12 03:37:21 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:37:21 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:37:21 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:37:21 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:37:21 user nova-compute[72081]:   </clock>
Apr 12 03:37:21 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:37:21 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:37:21 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:37:21 user nova-compute[72081]:   </cpu>
Apr 12 03:37:21 user nova-compute[72081]:   <devices>
Apr 12 03:37:21 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:37:21 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:37:21 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk"/>
Apr 12 03:37:21 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:37:21 user nova-compute[72081]:     </disk>
Apr 12 03:37:21 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:37:21 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:37:21 user nova-compute[72081]:       <source dev="/dev/sdd"/>
Apr 12 03:37:21 user nova-compute[72081]:       <target dev="vdb" bus="virtio"/>
Apr 12 03:37:21 user nova-compute[72081]:       <serial>0ca0118b-5d36-40b7-bb39-806756144b3a</serial>
Apr 12 03:37:21 user nova-compute[72081]:     </disk>
Apr 12 03:37:21 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:37:21 user nova-compute[72081]:       <mac address="fa:16:3e:b3:c5:ca"/>
Apr 12 03:37:21 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:37:21 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:37:21 user nova-compute[72081]:       <target dev="tap18f9f68c-ea"/>
Apr 12 03:37:21 user nova-compute[72081]:     </interface>
Apr 12 03:37:21 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:37:21 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/console.log" append="off"/>
Apr 12 03:37:21 user nova-compute[72081]:     </serial>
Apr 12 03:37:21 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:37:21 user nova-compute[72081]:     <video>
Apr 12 03:37:21 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:37:21 user nova-compute[72081]:     </video>
Apr 12 03:37:21 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:37:21 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:37:21 user nova-compute[72081]:     </rng>
Apr 12 03:37:21 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:37:21 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:37:21 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:37:21 user nova-compute[72081]:     </memballoon>
Apr 12 03:37:21 user nova-compute[72081]:   </devices>
Apr 12 03:37:21 user nova-compute[72081]: </domain>
Apr 12 03:37:21 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:37:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-12T03:33:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-2050314981',display_name='tempest-ServerActionsTestJSON-server-1847743592',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-2050314981',id=14,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKmrI9wy1CUIwhlKuXvzk0mD7Pt5nZFvI1CsqS0do6Lk76JQOvGl5lvx9rkz7UAM9WGocGXyWSOUOuKTtYZRD2HqKUbTN4VPOoliE1yuu+s0lx7vuZu45Mj7wDb2S8Ksyw==',key_name='tempest-keypair-364276420',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:33:27Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={rebuild='server'},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='815b65aa5ad9442db0a087c646f96bbd',ramdisk_id='',reservation_id='r-usz98205',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestJSON-1898059859',owner_user_name='tempest-ServerActionsTestJSON-1898059859-project-member'},tags=<?>,task_state='rebuild_spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:37:15Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a7f422f1a05a4a92ba387e424850b360',uuid=48977eb2-ce43-4577-a1cb-340e35703dbd,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:37:21 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converting VIF {"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:37:21 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:37:21 user nova-compute[72081]: DEBUG os_vif [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Plugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:37:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:37:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:37:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap18f9f68c-ea, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:37:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap18f9f68c-ea, col_values=(('external_ids', {'iface-id': '18f9f68c-eadd-4582-8f16-b88a13496852', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:b3:c5:ca', 'vm-uuid': '48977eb2-ce43-4577-a1cb-340e35703dbd'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:37:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:37:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:21 user nova-compute[72081]: INFO os_vif [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Successfully plugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea')
Apr 12 03:37:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:37:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:37:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] No VIF found with MAC fa:16:3e:b3:c5:ca, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:37:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:23 user nova-compute[72081]: DEBUG nova.compute.manager [req-7b18dd4d-e9c4-45b9-ba15-671baefb81e6 req-3ee94319-cfc0-4aa6-9862-4d18e8373236 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:37:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7b18dd4d-e9c4-45b9-ba15-671baefb81e6 req-3ee94319-cfc0-4aa6-9862-4d18e8373236 service nova] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7b18dd4d-e9c4-45b9-ba15-671baefb81e6 req-3ee94319-cfc0-4aa6-9862-4d18e8373236 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7b18dd4d-e9c4-45b9-ba15-671baefb81e6 req-3ee94319-cfc0-4aa6-9862-4d18e8373236 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:23 user nova-compute[72081]: DEBUG nova.compute.manager [req-7b18dd4d-e9c4-45b9-ba15-671baefb81e6 req-3ee94319-cfc0-4aa6-9862-4d18e8373236 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] No waiting events found dispatching network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:37:23 user nova-compute[72081]: WARNING nova.compute.manager [req-7b18dd4d-e9c4-45b9-ba15-671baefb81e6 req-3ee94319-cfc0-4aa6-9862-4d18e8373236 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received unexpected event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 for instance with vm_state active and task_state rebuild_spawning.
Apr 12 03:37:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:25 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Removed pending event for 48977eb2-ce43-4577-a1cb-340e35703dbd due to event {{(pid=72081) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 12 03:37:25 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270645.1596425, 48977eb2-ce43-4577-a1cb-340e35703dbd => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:37:25 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] VM Resumed (Lifecycle Event)
Apr 12 03:37:25 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:37:25 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:37:25 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Instance spawned successfully.
Apr 12 03:37:25 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:37:25 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:37:25 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rebuild_spawning, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:37:25 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:37:25 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:37:25 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:37:25 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:37:25 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:37:25 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:37:25 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] During sync_power_state the instance has a pending task (rebuild_spawning). Skip.
Apr 12 03:37:25 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270645.1630366, 48977eb2-ce43-4577-a1cb-340e35703dbd => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:37:25 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] VM Started (Lifecycle Event)
Apr 12 03:37:25 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:37:25 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: rebuild_spawning, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:37:25 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] During sync_power_state the instance has a pending task (rebuild_spawning). Skip.
Apr 12 03:37:25 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:37:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.finish_evacuation" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.finish_evacuation" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:25 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Trying to apply a migration context that does not seem to be set for this instance {{(pid=72081) apply_migration_context /opt/stack/nova/nova/objects/instance.py:1032}}
Apr 12 03:37:25 user nova-compute[72081]: DEBUG nova.compute.manager [req-ba4996c3-c5c6-4204-84da-f141e010f71e req-5db1c070-31f8-4a65-835a-0395b4384522 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:37:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ba4996c3-c5c6-4204-84da-f141e010f71e req-5db1c070-31f8-4a65-835a-0395b4384522 service nova] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ba4996c3-c5c6-4204-84da-f141e010f71e req-5db1c070-31f8-4a65-835a-0395b4384522 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ba4996c3-c5c6-4204-84da-f141e010f71e req-5db1c070-31f8-4a65-835a-0395b4384522 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:25 user nova-compute[72081]: DEBUG nova.compute.manager [req-ba4996c3-c5c6-4204-84da-f141e010f71e req-5db1c070-31f8-4a65-835a-0395b4384522 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] No waiting events found dispatching network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:37:25 user nova-compute[72081]: WARNING nova.compute.manager [req-ba4996c3-c5c6-4204-84da-f141e010f71e req-5db1c070-31f8-4a65-835a-0395b4384522 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received unexpected event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 for instance with vm_state active and task_state None.
Apr 12 03:37:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4e14b227-2d89-457b-a704-f5c851caba6e tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.finish_evacuation" :: held 0.093s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d97fac59-4f29-4f08-8ae9-ab36499200ff tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Acquiring lock "72ba1218-065d-40c4-8503-7c4155e47503" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d97fac59-4f29-4f08-8ae9-ab36499200ff tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "72ba1218-065d-40c4-8503-7c4155e47503" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d97fac59-4f29-4f08-8ae9-ab36499200ff tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Acquiring lock "72ba1218-065d-40c4-8503-7c4155e47503-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d97fac59-4f29-4f08-8ae9-ab36499200ff tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d97fac59-4f29-4f08-8ae9-ab36499200ff tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:27 user nova-compute[72081]: INFO nova.compute.manager [None req-d97fac59-4f29-4f08-8ae9-ab36499200ff tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Terminating instance
Apr 12 03:37:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-d97fac59-4f29-4f08-8ae9-ab36499200ff tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:37:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-7b5d1b42-70e3-42e5-99f6-9c62bc5d5cd6 req-ec10a11b-9ca0-4e27-a258-acbe2fbe5a90 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received event network-vif-unplugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:37:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7b5d1b42-70e3-42e5-99f6-9c62bc5d5cd6 req-ec10a11b-9ca0-4e27-a258-acbe2fbe5a90 service nova] Acquiring lock "72ba1218-065d-40c4-8503-7c4155e47503-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7b5d1b42-70e3-42e5-99f6-9c62bc5d5cd6 req-ec10a11b-9ca0-4e27-a258-acbe2fbe5a90 service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7b5d1b42-70e3-42e5-99f6-9c62bc5d5cd6 req-ec10a11b-9ca0-4e27-a258-acbe2fbe5a90 service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-7b5d1b42-70e3-42e5-99f6-9c62bc5d5cd6 req-ec10a11b-9ca0-4e27-a258-acbe2fbe5a90 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] No waiting events found dispatching network-vif-unplugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:37:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-7b5d1b42-70e3-42e5-99f6-9c62bc5d5cd6 req-ec10a11b-9ca0-4e27-a258-acbe2fbe5a90 service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received event network-vif-unplugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:37:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:28 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Instance destroyed successfully.
Apr 12 03:37:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:28 user nova-compute[72081]: DEBUG nova.objects.instance [None req-d97fac59-4f29-4f08-8ae9-ab36499200ff tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lazy-loading 'resources' on Instance uuid 72ba1218-065d-40c4-8503-7c4155e47503 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:37:28 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-d97fac59-4f29-4f08-8ae9-ab36499200ff tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:30:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerStableDeviceRescueTest-server-252473144',display_name='tempest-ServerStableDeviceRescueTest-server-252473144',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverstabledevicerescuetest-server-252473144',id=5,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBInWJADLpni9D9TfgnC1+/LMPrrQuD/lxuqixLR+mL8FYE3064UBREFALqAw20h2nHoOSsmv2cMRVuFj3Oi+UewdLxZXOi4pdekw/Cvt3tr0uQWDcvZgNGpzrP2V/qty5w==',key_name='tempest-keypair-31851495',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:34:22Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='a027d7708e6747fe970c388d6ba66359',ramdisk_id='',reservation_id='r-30kyz70u',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerStableDeviceRescueTest-744520899',owner_user_name='tempest-ServerStableDeviceRescueTest-744520899-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:36:08Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='9405e57e55e84976b0caeba84e3aef62',uuid=72ba1218-065d-40c4-8503-7c4155e47503,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "address": "fa:16:3e:c0:f6:ee", "network": {"id": "b4464ad0-6022-43e7-bfe4-229e6a3a0600", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1704232678-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.188", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a027d7708e6747fe970c388d6ba66359", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcff588d4-f4", "ovs_interfaceid": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:37:28 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-d97fac59-4f29-4f08-8ae9-ab36499200ff tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Converting VIF {"id": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "address": "fa:16:3e:c0:f6:ee", "network": {"id": "b4464ad0-6022-43e7-bfe4-229e6a3a0600", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1704232678-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.188", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a027d7708e6747fe970c388d6ba66359", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcff588d4-f4", "ovs_interfaceid": "cff588d4-f46a-4d44-a3e6-1c04f6abd380", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:37:28 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-d97fac59-4f29-4f08-8ae9-ab36499200ff tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:c0:f6:ee,bridge_name='br-int',has_traffic_filtering=True,id=cff588d4-f46a-4d44-a3e6-1c04f6abd380,network=Network(b4464ad0-6022-43e7-bfe4-229e6a3a0600),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapcff588d4-f4') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:37:28 user nova-compute[72081]: DEBUG os_vif [None req-d97fac59-4f29-4f08-8ae9-ab36499200ff tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:c0:f6:ee,bridge_name='br-int',has_traffic_filtering=True,id=cff588d4-f46a-4d44-a3e6-1c04f6abd380,network=Network(b4464ad0-6022-43e7-bfe4-229e6a3a0600),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapcff588d4-f4') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:37:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapcff588d4-f4, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:37:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:37:28 user nova-compute[72081]: INFO os_vif [None req-d97fac59-4f29-4f08-8ae9-ab36499200ff tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:c0:f6:ee,bridge_name='br-int',has_traffic_filtering=True,id=cff588d4-f46a-4d44-a3e6-1c04f6abd380,network=Network(b4464ad0-6022-43e7-bfe4-229e6a3a0600),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapcff588d4-f4')
Apr 12 03:37:28 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-d97fac59-4f29-4f08-8ae9-ab36499200ff tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Deleting instance files /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503_del
Apr 12 03:37:28 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-d97fac59-4f29-4f08-8ae9-ab36499200ff tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Deletion of /opt/stack/data/nova/instances/72ba1218-065d-40c4-8503-7c4155e47503_del complete
Apr 12 03:37:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:28 user nova-compute[72081]: INFO nova.compute.manager [None req-d97fac59-4f29-4f08-8ae9-ab36499200ff tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Took 0.95 seconds to destroy the instance on the hypervisor.
Apr 12 03:37:28 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-d97fac59-4f29-4f08-8ae9-ab36499200ff tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:37:28 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:37:28 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:37:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:37:30 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Took 1.28 seconds to deallocate network for instance.
Apr 12 03:37:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d97fac59-4f29-4f08-8ae9-ab36499200ff tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d97fac59-4f29-4f08-8ae9-ab36499200ff tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received event network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] Acquiring lock "72ba1218-065d-40c4-8503-7c4155e47503-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] No waiting events found dispatching network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:37:30 user nova-compute[72081]: WARNING nova.compute.manager [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received unexpected event network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 for instance with vm_state deleted and task_state None.
Apr 12 03:37:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received event network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] Acquiring lock "72ba1218-065d-40c4-8503-7c4155e47503-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] No waiting events found dispatching network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:37:30 user nova-compute[72081]: WARNING nova.compute.manager [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received unexpected event network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 for instance with vm_state deleted and task_state None.
Apr 12 03:37:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received event network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] Acquiring lock "72ba1218-065d-40c4-8503-7c4155e47503-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] No waiting events found dispatching network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:37:30 user nova-compute[72081]: WARNING nova.compute.manager [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received unexpected event network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 for instance with vm_state deleted and task_state None.
Apr 12 03:37:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received event network-vif-unplugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] Acquiring lock "72ba1218-065d-40c4-8503-7c4155e47503-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] No waiting events found dispatching network-vif-unplugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:37:30 user nova-compute[72081]: WARNING nova.compute.manager [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received unexpected event network-vif-unplugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 for instance with vm_state deleted and task_state None.
Apr 12 03:37:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received event network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] Acquiring lock "72ba1218-065d-40c4-8503-7c4155e47503-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] Lock "72ba1218-065d-40c4-8503-7c4155e47503-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] No waiting events found dispatching network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:37:30 user nova-compute[72081]: WARNING nova.compute.manager [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received unexpected event network-vif-plugged-cff588d4-f46a-4d44-a3e6-1c04f6abd380 for instance with vm_state deleted and task_state None.
Apr 12 03:37:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-4b56003e-ddb5-4b24-92ce-6d124c783671 req-2e963d54-5f7a-4167-b6df-d8fa21a22dfd service nova] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Received event network-vif-deleted-cff588d4-f46a-4d44-a3e6-1c04f6abd380 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-d97fac59-4f29-4f08-8ae9-ab36499200ff tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-d97fac59-4f29-4f08-8ae9-ab36499200ff tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:37:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d97fac59-4f29-4f08-8ae9-ab36499200ff tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.279s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:30 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-d97fac59-4f29-4f08-8ae9-ab36499200ff tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Deleted allocations for instance 72ba1218-065d-40c4-8503-7c4155e47503
Apr 12 03:37:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d97fac59-4f29-4f08-8ae9-ab36499200ff tempest-ServerStableDeviceRescueTest-744520899 tempest-ServerStableDeviceRescueTest-744520899-project-member] Lock "72ba1218-065d-40c4-8503-7c4155e47503" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.702s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "06481254-722a-4053-a6af-11f5dcc68636" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "06481254-722a-4053-a6af-11f5dcc68636" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:31 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:37:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.004s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:31 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:37:31 user nova-compute[72081]: INFO nova.compute.claims [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Claim successful on node user
Apr 12 03:37:31 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:37:31 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:37:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.335s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:31 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:37:31 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:37:31 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:37:31 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:37:31 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:37:31 user nova-compute[72081]: DEBUG nova.policy [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'de1fe5db01624224ab7b615c908d990b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '738ebb6b798c4974aa46f9bbe3828279', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:37:32 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Creating image(s)
Apr 12 03:37:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "/opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "/opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "/opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.141s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.160s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk 1073741824" returned: 0 in 0.051s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.217s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.150s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Checking if we can resize image /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Cannot resize image /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG nova.objects.instance [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lazy-loading 'migration_context' on Instance uuid 06481254-722a-4053-a6af-11f5dcc68636 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Ensure instance console log exists: /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:32 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Successfully created port: 5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:37:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:33 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Successfully updated port: 5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:37:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "refresh_cache-06481254-722a-4053-a6af-11f5dcc68636" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:37:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquired lock "refresh_cache-06481254-722a-4053-a6af-11f5dcc68636" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:37:33 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:37:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-fbfecbcf-46e6-46a5-9e8d-a72a36616c8e req-115b93f0-56bd-4fd2-8243-12ab438209d0 service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Received event network-changed-5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:37:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-fbfecbcf-46e6-46a5-9e8d-a72a36616c8e req-115b93f0-56bd-4fd2-8243-12ab438209d0 service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Refreshing instance network info cache due to event network-changed-5861aafc-4312-4d82-965c-2d1d993d8ea0. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:37:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-fbfecbcf-46e6-46a5-9e8d-a72a36616c8e req-115b93f0-56bd-4fd2-8243-12ab438209d0 service nova] Acquiring lock "refresh_cache-06481254-722a-4053-a6af-11f5dcc68636" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:37:33 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Updating instance_info_cache with network_info: [{"id": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "address": "fa:16:3e:c7:7c:eb", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5861aafc-43", "ovs_interfaceid": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Releasing lock "refresh_cache-06481254-722a-4053-a6af-11f5dcc68636" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Instance network_info: |[{"id": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "address": "fa:16:3e:c7:7c:eb", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5861aafc-43", "ovs_interfaceid": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-fbfecbcf-46e6-46a5-9e8d-a72a36616c8e req-115b93f0-56bd-4fd2-8243-12ab438209d0 service nova] Acquired lock "refresh_cache-06481254-722a-4053-a6af-11f5dcc68636" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.network.neutron [req-fbfecbcf-46e6-46a5-9e8d-a72a36616c8e req-115b93f0-56bd-4fd2-8243-12ab438209d0 service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Refreshing network info cache for port 5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Start _get_guest_xml network_info=[{"id": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "address": "fa:16:3e:c7:7c:eb", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5861aafc-43", "ovs_interfaceid": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:37:34 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:37:34 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:37:30Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-405065337',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-405065337',id=20,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEV0Ykds0yE0AtGhC79nOfa8CiixAX+fWYFS4r0KbDrdX/eto3bHX8KjkQpgbhQK4OVD9q33q1+I1pg47ZtrC8YTjrFvz5r71kp0Qd9KtUm3DHiDVGrOt+n/RWN+EIIG+w==',key_name='tempest-keypair-2125155953',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='738ebb6b798c4974aa46f9bbe3828279',ramdisk_id='',reservation_id='r-jbxvhjdl',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeShelveTestJSON-2139975165',owner_user_name='tempest-AttachVolumeShelveTestJSON-2139975165-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:37:32Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='de1fe5db01624224ab7b615c908d990b',uuid=06481254-722a-4053-a6af-11f5dcc68636,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "address": "fa:16:3e:c7:7c:eb", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5861aafc-43", "ovs_interfaceid": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Converting VIF {"id": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "address": "fa:16:3e:c7:7c:eb", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5861aafc-43", "ovs_interfaceid": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:c7:7c:eb,bridge_name='br-int',has_traffic_filtering=True,id=5861aafc-4312-4d82-965c-2d1d993d8ea0,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5861aafc-43') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.objects.instance [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lazy-loading 'pci_devices' on Instance uuid 06481254-722a-4053-a6af-11f5dcc68636 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:37:34 user nova-compute[72081]:   <uuid>06481254-722a-4053-a6af-11f5dcc68636</uuid>
Apr 12 03:37:34 user nova-compute[72081]:   <name>instance-00000014</name>
Apr 12 03:37:34 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:37:34 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:37:34 user nova-compute[72081]:   <metadata>
Apr 12 03:37:34 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:37:34 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:37:34 user nova-compute[72081]:       <nova:name>tempest-AttachVolumeShelveTestJSON-server-405065337</nova:name>
Apr 12 03:37:34 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:37:34</nova:creationTime>
Apr 12 03:37:34 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:37:34 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:37:34 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:37:34 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:37:34 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:37:34 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:37:34 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:37:34 user nova-compute[72081]:       <nova:owner>
Apr 12 03:37:34 user nova-compute[72081]:         <nova:user uuid="de1fe5db01624224ab7b615c908d990b">tempest-AttachVolumeShelveTestJSON-2139975165-project-member</nova:user>
Apr 12 03:37:34 user nova-compute[72081]:         <nova:project uuid="738ebb6b798c4974aa46f9bbe3828279">tempest-AttachVolumeShelveTestJSON-2139975165</nova:project>
Apr 12 03:37:34 user nova-compute[72081]:       </nova:owner>
Apr 12 03:37:34 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:37:34 user nova-compute[72081]:       <nova:ports>
Apr 12 03:37:34 user nova-compute[72081]:         <nova:port uuid="5861aafc-4312-4d82-965c-2d1d993d8ea0">
Apr 12 03:37:34 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Apr 12 03:37:34 user nova-compute[72081]:         </nova:port>
Apr 12 03:37:34 user nova-compute[72081]:       </nova:ports>
Apr 12 03:37:34 user nova-compute[72081]:     </nova:instance>
Apr 12 03:37:34 user nova-compute[72081]:   </metadata>
Apr 12 03:37:34 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:37:34 user nova-compute[72081]:     <system>
Apr 12 03:37:34 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:37:34 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:37:34 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:37:34 user nova-compute[72081]:       <entry name="serial">06481254-722a-4053-a6af-11f5dcc68636</entry>
Apr 12 03:37:34 user nova-compute[72081]:       <entry name="uuid">06481254-722a-4053-a6af-11f5dcc68636</entry>
Apr 12 03:37:34 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:37:34 user nova-compute[72081]:     </system>
Apr 12 03:37:34 user nova-compute[72081]:   </sysinfo>
Apr 12 03:37:34 user nova-compute[72081]:   <os>
Apr 12 03:37:34 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:37:34 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:37:34 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:37:34 user nova-compute[72081]:   </os>
Apr 12 03:37:34 user nova-compute[72081]:   <features>
Apr 12 03:37:34 user nova-compute[72081]:     <acpi/>
Apr 12 03:37:34 user nova-compute[72081]:     <apic/>
Apr 12 03:37:34 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:37:34 user nova-compute[72081]:   </features>
Apr 12 03:37:34 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:37:34 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:37:34 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:37:34 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:37:34 user nova-compute[72081]:   </clock>
Apr 12 03:37:34 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:37:34 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:37:34 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:37:34 user nova-compute[72081]:   </cpu>
Apr 12 03:37:34 user nova-compute[72081]:   <devices>
Apr 12 03:37:34 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:37:34 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:37:34 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk"/>
Apr 12 03:37:34 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:37:34 user nova-compute[72081]:     </disk>
Apr 12 03:37:34 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:37:34 user nova-compute[72081]:       <mac address="fa:16:3e:c7:7c:eb"/>
Apr 12 03:37:34 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:37:34 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:37:34 user nova-compute[72081]:       <target dev="tap5861aafc-43"/>
Apr 12 03:37:34 user nova-compute[72081]:     </interface>
Apr 12 03:37:34 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:37:34 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/console.log" append="off"/>
Apr 12 03:37:34 user nova-compute[72081]:     </serial>
Apr 12 03:37:34 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:37:34 user nova-compute[72081]:     <video>
Apr 12 03:37:34 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:37:34 user nova-compute[72081]:     </video>
Apr 12 03:37:34 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:37:34 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:37:34 user nova-compute[72081]:     </rng>
Apr 12 03:37:34 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:37:34 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:37:34 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:37:34 user nova-compute[72081]:     </memballoon>
Apr 12 03:37:34 user nova-compute[72081]:   </devices>
Apr 12 03:37:34 user nova-compute[72081]: </domain>
Apr 12 03:37:34 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:37:30Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-405065337',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-405065337',id=20,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEV0Ykds0yE0AtGhC79nOfa8CiixAX+fWYFS4r0KbDrdX/eto3bHX8KjkQpgbhQK4OVD9q33q1+I1pg47ZtrC8YTjrFvz5r71kp0Qd9KtUm3DHiDVGrOt+n/RWN+EIIG+w==',key_name='tempest-keypair-2125155953',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='738ebb6b798c4974aa46f9bbe3828279',ramdisk_id='',reservation_id='r-jbxvhjdl',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeShelveTestJSON-2139975165',owner_user_name='tempest-AttachVolumeShelveTestJSON-2139975165-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:37:32Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='de1fe5db01624224ab7b615c908d990b',uuid=06481254-722a-4053-a6af-11f5dcc68636,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "address": "fa:16:3e:c7:7c:eb", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5861aafc-43", "ovs_interfaceid": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Converting VIF {"id": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "address": "fa:16:3e:c7:7c:eb", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5861aafc-43", "ovs_interfaceid": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:c7:7c:eb,bridge_name='br-int',has_traffic_filtering=True,id=5861aafc-4312-4d82-965c-2d1d993d8ea0,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5861aafc-43') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG os_vif [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:c7:7c:eb,bridge_name='br-int',has_traffic_filtering=True,id=5861aafc-4312-4d82-965c-2d1d993d8ea0,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5861aafc-43') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap5861aafc-43, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap5861aafc-43, col_values=(('external_ids', {'iface-id': '5861aafc-4312-4d82-965c-2d1d993d8ea0', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:c7:7c:eb', 'vm-uuid': '06481254-722a-4053-a6af-11f5dcc68636'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:34 user nova-compute[72081]: INFO os_vif [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:c7:7c:eb,bridge_name='br-int',has_traffic_filtering=True,id=5861aafc-4312-4d82-965c-2d1d993d8ea0,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5861aafc-43')
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] No VIF found with MAC fa:16:3e:c7:7c:eb, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.network.neutron [req-fbfecbcf-46e6-46a5-9e8d-a72a36616c8e req-115b93f0-56bd-4fd2-8243-12ab438209d0 service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Updated VIF entry in instance network info cache for port 5861aafc-4312-4d82-965c-2d1d993d8ea0. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG nova.network.neutron [req-fbfecbcf-46e6-46a5-9e8d-a72a36616c8e req-115b93f0-56bd-4fd2-8243-12ab438209d0 service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Updating instance_info_cache with network_info: [{"id": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "address": "fa:16:3e:c7:7c:eb", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5861aafc-43", "ovs_interfaceid": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:37:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-fbfecbcf-46e6-46a5-9e8d-a72a36616c8e req-115b93f0-56bd-4fd2-8243-12ab438209d0 service nova] Releasing lock "refresh_cache-06481254-722a-4053-a6af-11f5dcc68636" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:37:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:35 user nova-compute[72081]: DEBUG nova.compute.manager [req-0c5897c4-7d34-4a7f-9073-bf9f6dd6f7a6 req-8c9f8444-002e-4058-88b0-d8d4b1a2a13c service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Received event network-vif-plugged-5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:37:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0c5897c4-7d34-4a7f-9073-bf9f6dd6f7a6 req-8c9f8444-002e-4058-88b0-d8d4b1a2a13c service nova] Acquiring lock "06481254-722a-4053-a6af-11f5dcc68636-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0c5897c4-7d34-4a7f-9073-bf9f6dd6f7a6 req-8c9f8444-002e-4058-88b0-d8d4b1a2a13c service nova] Lock "06481254-722a-4053-a6af-11f5dcc68636-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0c5897c4-7d34-4a7f-9073-bf9f6dd6f7a6 req-8c9f8444-002e-4058-88b0-d8d4b1a2a13c service nova] Lock "06481254-722a-4053-a6af-11f5dcc68636-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:35 user nova-compute[72081]: DEBUG nova.compute.manager [req-0c5897c4-7d34-4a7f-9073-bf9f6dd6f7a6 req-8c9f8444-002e-4058-88b0-d8d4b1a2a13c service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] No waiting events found dispatching network-vif-plugged-5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:37:35 user nova-compute[72081]: WARNING nova.compute.manager [req-0c5897c4-7d34-4a7f-9073-bf9f6dd6f7a6 req-8c9f8444-002e-4058-88b0-d8d4b1a2a13c service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Received unexpected event network-vif-plugged-5861aafc-4312-4d82-965c-2d1d993d8ea0 for instance with vm_state building and task_state spawning.
Apr 12 03:37:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:37 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270657.7656755, 06481254-722a-4053-a6af-11f5dcc68636 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:37:37 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 06481254-722a-4053-a6af-11f5dcc68636] VM Resumed (Lifecycle Event)
Apr 12 03:37:37 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:37:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:37:37 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Instance spawned successfully.
Apr 12 03:37:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:37:37 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:37:37 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:37:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:37:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:37:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:37:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:37:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:37:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:37:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-64a2bd2f-0a0a-408a-b3c2-e90cc05a913d req-126bbedb-7e6b-46cd-a771-844bde80d373 service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Received event network-vif-plugged-5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:37:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-64a2bd2f-0a0a-408a-b3c2-e90cc05a913d req-126bbedb-7e6b-46cd-a771-844bde80d373 service nova] Acquiring lock "06481254-722a-4053-a6af-11f5dcc68636-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-64a2bd2f-0a0a-408a-b3c2-e90cc05a913d req-126bbedb-7e6b-46cd-a771-844bde80d373 service nova] Lock "06481254-722a-4053-a6af-11f5dcc68636-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-64a2bd2f-0a0a-408a-b3c2-e90cc05a913d req-126bbedb-7e6b-46cd-a771-844bde80d373 service nova] Lock "06481254-722a-4053-a6af-11f5dcc68636-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-64a2bd2f-0a0a-408a-b3c2-e90cc05a913d req-126bbedb-7e6b-46cd-a771-844bde80d373 service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] No waiting events found dispatching network-vif-plugged-5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:37:37 user nova-compute[72081]: WARNING nova.compute.manager [req-64a2bd2f-0a0a-408a-b3c2-e90cc05a913d req-126bbedb-7e6b-46cd-a771-844bde80d373 service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Received unexpected event network-vif-plugged-5861aafc-4312-4d82-965c-2d1d993d8ea0 for instance with vm_state building and task_state spawning.
Apr 12 03:37:37 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 06481254-722a-4053-a6af-11f5dcc68636] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:37:37 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270657.7702036, 06481254-722a-4053-a6af-11f5dcc68636 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:37:37 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 06481254-722a-4053-a6af-11f5dcc68636] VM Started (Lifecycle Event)
Apr 12 03:37:37 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:37:37 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:37:37 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 06481254-722a-4053-a6af-11f5dcc68636] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:37:37 user nova-compute[72081]: INFO nova.compute.manager [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Took 5.85 seconds to spawn the instance on the hypervisor.
Apr 12 03:37:37 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:37:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:37 user nova-compute[72081]: INFO nova.compute.manager [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Took 6.54 seconds to build instance.
Apr 12 03:37:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8f6278c8-d7c6-49f7-91e2-8af6d58a32f3 tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "06481254-722a-4053-a6af-11f5dcc68636" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.637s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:43 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681270648.5921333, 72ba1218-065d-40c4-8503-7c4155e47503 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:37:43 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] VM Stopped (Lifecycle Event)
Apr 12 03:37:43 user nova-compute[72081]: DEBUG nova.compute.manager [None req-dd33c0d5-95d2-412f-ae55-d192119ea3f8 None None] [instance: 72ba1218-065d-40c4-8503-7c4155e47503] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:37:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:37:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 03:37:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 03:37:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 03:37:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "1eef2c91-dfad-43aa-b193-25970db83b79" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "1eef2c91-dfad-43aa-b193-25970db83b79" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:53 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:37:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:53 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:37:53 user nova-compute[72081]: INFO nova.compute.claims [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Claim successful on node user
Apr 12 03:37:53 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:37:53 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:37:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.334s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:53 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:37:53 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:37:53 user nova-compute[72081]: DEBUG nova.network.neutron [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:37:53 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:37:53 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:37:53 user nova-compute[72081]: INFO nova.virt.block_device [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Booting with volume fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf at /dev/vda
Apr 12 03:37:53 user nova-compute[72081]: DEBUG os_brick.utils [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:37:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2c2be2fa-fe2d-41e2-9e0c-a1374ed9e8f8]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:53 user nova-compute[72081]: DEBUG nova.policy [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'b00b348933224558aad1cb12921f100f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '003688353e2f4b36831c574ebaf65173', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:37:54 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:54 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[04cf9d20-fb75-4a5f-8a14-85c4ebfdc2ff]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:54 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:54 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:54 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ea135e2-a785-4d6d-bfda-e4dd8e90928a]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:54 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8a22428b-4987-4330-9721-f7736e309327]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:54 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:54 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:37:54 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:37:54 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:54 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:37:54 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:37:54 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3f7385a5-dc13-4cb0-9dda-4479e838b8c5]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:54 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:37:54 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:37:54 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:37:54 user nova-compute[72081]: DEBUG os_brick.utils [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] <== get_connector_properties: return (126ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:37:54 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Updating existing volume attachment record: d73542e1-4e38-4d1d-945b-d0192d644bd8 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:37:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:54 user nova-compute[72081]: DEBUG nova.network.neutron [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Successfully created port: 50b78ba6-f017-4643-a2ec-fa0f348bb270 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:37:55 user nova-compute[72081]: DEBUG nova.network.neutron [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Successfully updated port: 50b78ba6-f017-4643-a2ec-fa0f348bb270 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:37:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "refresh_cache-1eef2c91-dfad-43aa-b193-25970db83b79" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:37:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquired lock "refresh_cache-1eef2c91-dfad-43aa-b193-25970db83b79" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:37:55 user nova-compute[72081]: DEBUG nova.network.neutron [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:37:55 user nova-compute[72081]: DEBUG nova.compute.manager [req-a747c40b-3563-4151-beae-108b033df07d req-86db0e4e-1b97-4d1c-b4f2-c7f7af860811 service nova] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Received event network-changed-50b78ba6-f017-4643-a2ec-fa0f348bb270 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:37:55 user nova-compute[72081]: DEBUG nova.compute.manager [req-a747c40b-3563-4151-beae-108b033df07d req-86db0e4e-1b97-4d1c-b4f2-c7f7af860811 service nova] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Refreshing instance network info cache due to event network-changed-50b78ba6-f017-4643-a2ec-fa0f348bb270. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:37:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a747c40b-3563-4151-beae-108b033df07d req-86db0e4e-1b97-4d1c-b4f2-c7f7af860811 service nova] Acquiring lock "refresh_cache-1eef2c91-dfad-43aa-b193-25970db83b79" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:37:55 user nova-compute[72081]: DEBUG nova.network.neutron [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:37:55 user nova-compute[72081]: DEBUG nova.network.neutron [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Updating instance_info_cache with network_info: [{"id": "50b78ba6-f017-4643-a2ec-fa0f348bb270", "address": "fa:16:3e:60:70:2a", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50b78ba6-f0", "ovs_interfaceid": "50b78ba6-f017-4643-a2ec-fa0f348bb270", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:37:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Releasing lock "refresh_cache-1eef2c91-dfad-43aa-b193-25970db83b79" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:37:55 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Instance network_info: |[{"id": "50b78ba6-f017-4643-a2ec-fa0f348bb270", "address": "fa:16:3e:60:70:2a", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50b78ba6-f0", "ovs_interfaceid": "50b78ba6-f017-4643-a2ec-fa0f348bb270", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:37:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a747c40b-3563-4151-beae-108b033df07d req-86db0e4e-1b97-4d1c-b4f2-c7f7af860811 service nova] Acquired lock "refresh_cache-1eef2c91-dfad-43aa-b193-25970db83b79" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:37:55 user nova-compute[72081]: DEBUG nova.network.neutron [req-a747c40b-3563-4151-beae-108b033df07d req-86db0e4e-1b97-4d1c-b4f2-c7f7af860811 service nova] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Refreshing network info cache for port 50b78ba6-f017-4643-a2ec-fa0f348bb270 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:37:56 user nova-compute[72081]: DEBUG nova.network.neutron [req-a747c40b-3563-4151-beae-108b033df07d req-86db0e4e-1b97-4d1c-b4f2-c7f7af860811 service nova] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Updated VIF entry in instance network info cache for port 50b78ba6-f017-4643-a2ec-fa0f348bb270. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:37:56 user nova-compute[72081]: DEBUG nova.network.neutron [req-a747c40b-3563-4151-beae-108b033df07d req-86db0e4e-1b97-4d1c-b4f2-c7f7af860811 service nova] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Updating instance_info_cache with network_info: [{"id": "50b78ba6-f017-4643-a2ec-fa0f348bb270", "address": "fa:16:3e:60:70:2a", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50b78ba6-f0", "ovs_interfaceid": "50b78ba6-f017-4643-a2ec-fa0f348bb270", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:37:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a747c40b-3563-4151-beae-108b033df07d req-86db0e4e-1b97-4d1c-b4f2-c7f7af860811 service nova] Releasing lock "refresh_cache-1eef2c91-dfad-43aa-b193-25970db83b79" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:37:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:37:58 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Creating image(s)
Apr 12 03:37:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Did not create local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Ensure instance console log exists: /opt/stack/data/nova/instances/1eef2c91-dfad-43aa-b193-25970db83b79/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Start _get_guest_xml network_info=[{"id": "50b78ba6-f017-4643-a2ec-fa0f348bb270", "address": "fa:16:3e:60:70:2a", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50b78ba6-f0", "ovs_interfaceid": "50b78ba6-f017-4643-a2ec-fa0f348bb270", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'zxOqldvX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '1eef2c91-dfad-43aa-b193-25970db83b79', 'attached_at': '', 'detached_at': '', 'volume_id': 'fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf', 'serial': 'fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf'}, 'disk_bus': 'virtio', 'delete_on_termination': True, 'boot_index': 0, 'guest_format': None, 'device_type': 'disk', 'attachment_id': 'd73542e1-4e38-4d1d-945b-d0192d644bd8', 'volume_type': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:37:58 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:37:58 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:37:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'zxOqldvX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:37:58 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:37:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:37:58 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:37:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:37:58 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d7331507-e682-448a-92f0-a49e0ec1da3c]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:37:58 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[352e3e05-2e49-4412-9278-5b1a0efed077]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf] added
Apr 12 03:37:58 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b55d0bda-c87c-4c85-ab9f-ff1c01c1600e]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[50e29b3b-066f-4511-84ac-f7458f59f09d]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf -p 172.16.0.220:3260 --op update -n node.session.auth.username -v zxOqldvX {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf -p 172.16.0.220:3260 --op update -n node.session.auth.username -v zxOqldvX" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2d7c130a-6306-405e-b8a2-3ec7898c05c1]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'zxOqldvX'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ae793e6f-7b5a-49b8-ae84-ea344a7efc53]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[fb88ff87-2b43-4854-93d9-6a7aa991c87e]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:37:58 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:37:58 user nova-compute[72081]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)
Apr 12 03:37:58 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:37:58 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:37:58 user nova-compute[72081]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)
Apr 12 03:37:58 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf -p 172.16.0.220:3260 --login" returned: 0 in 0.061s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0860a89d-4ec0-4757-95c9-f67b557483ae]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf, portal: 172.16.0.220,3260]
Apr 12 03:37:58 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf, portal: 172.16.0.220,3260] successful.
Apr 12 03:37:58 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.028s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0726d375-fbde-43f4-be3b-323c6452e60d]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d5c1595d-2257-41f2-90a6-f75f89b8f07a]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:37:58 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:37:58 user nova-compute[72081]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)
Apr 12 03:37:58 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:37:58 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:37:58 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:37:58 user nova-compute[72081]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)
Apr 12 03:37:58 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:37:58 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.302s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] HCTL ('36', '-', '-', 0) found on session 25 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[dd995337-6808-4c3a-85eb-901da5863b04]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Searching for a device in session 25 and hctl ['36', '*', '*', 0] yield: sde {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:37:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'zxOqldvX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:37:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.341s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:37:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] <== connect_volume: return (1342ms) {'type': 'block', 'scsi_wwn': '23962633965376365', 'path': '/dev/sde'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:37:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23962633965376365', 'path': '/dev/sde'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:37:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:37:52Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-volume-backed-server-966703655',display_name='tempest-TestVolumeBootPattern-volume-backed-server-966703655',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-volume-backed-server-966703655',id=21,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='003688353e2f4b36831c574ebaf65173',ramdisk_id='',reservation_id='r-ux4uytew',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-235043461',owner_user_name='tempest-TestVolumeBootPattern-235043461-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:37:54Z,user_data=None,user_id='b00b348933224558aad1cb12921f100f',uuid=1eef2c91-dfad-43aa-b193-25970db83b79,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "50b78ba6-f017-4643-a2ec-fa0f348bb270", "address": "fa:16:3e:60:70:2a", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50b78ba6-f0", "ovs_interfaceid": "50b78ba6-f017-4643-a2ec-fa0f348bb270", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:37:59 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converting VIF {"id": "50b78ba6-f017-4643-a2ec-fa0f348bb270", "address": "fa:16:3e:60:70:2a", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50b78ba6-f0", "ovs_interfaceid": "50b78ba6-f017-4643-a2ec-fa0f348bb270", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:37:59 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:60:70:2a,bridge_name='br-int',has_traffic_filtering=True,id=50b78ba6-f017-4643-a2ec-fa0f348bb270,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap50b78ba6-f0') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:37:59 user nova-compute[72081]: DEBUG nova.objects.instance [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lazy-loading 'pci_devices' on Instance uuid 1eef2c91-dfad-43aa-b193-25970db83b79 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:37:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:37:59 user nova-compute[72081]:   <uuid>1eef2c91-dfad-43aa-b193-25970db83b79</uuid>
Apr 12 03:37:59 user nova-compute[72081]:   <name>instance-00000015</name>
Apr 12 03:37:59 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:37:59 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:37:59 user nova-compute[72081]:   <metadata>
Apr 12 03:37:59 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:37:59 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:37:59 user nova-compute[72081]:       <nova:name>tempest-TestVolumeBootPattern-volume-backed-server-966703655</nova:name>
Apr 12 03:37:59 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:37:58</nova:creationTime>
Apr 12 03:37:59 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:37:59 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:37:59 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:37:59 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:37:59 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:37:59 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:37:59 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:37:59 user nova-compute[72081]:       <nova:owner>
Apr 12 03:37:59 user nova-compute[72081]:         <nova:user uuid="b00b348933224558aad1cb12921f100f">tempest-TestVolumeBootPattern-235043461-project-member</nova:user>
Apr 12 03:37:59 user nova-compute[72081]:         <nova:project uuid="003688353e2f4b36831c574ebaf65173">tempest-TestVolumeBootPattern-235043461</nova:project>
Apr 12 03:37:59 user nova-compute[72081]:       </nova:owner>
Apr 12 03:37:59 user nova-compute[72081]:       <nova:ports>
Apr 12 03:37:59 user nova-compute[72081]:         <nova:port uuid="50b78ba6-f017-4643-a2ec-fa0f348bb270">
Apr 12 03:37:59 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 12 03:37:59 user nova-compute[72081]:         </nova:port>
Apr 12 03:37:59 user nova-compute[72081]:       </nova:ports>
Apr 12 03:37:59 user nova-compute[72081]:     </nova:instance>
Apr 12 03:37:59 user nova-compute[72081]:   </metadata>
Apr 12 03:37:59 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:37:59 user nova-compute[72081]:     <system>
Apr 12 03:37:59 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:37:59 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:37:59 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:37:59 user nova-compute[72081]:       <entry name="serial">1eef2c91-dfad-43aa-b193-25970db83b79</entry>
Apr 12 03:37:59 user nova-compute[72081]:       <entry name="uuid">1eef2c91-dfad-43aa-b193-25970db83b79</entry>
Apr 12 03:37:59 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:37:59 user nova-compute[72081]:     </system>
Apr 12 03:37:59 user nova-compute[72081]:   </sysinfo>
Apr 12 03:37:59 user nova-compute[72081]:   <os>
Apr 12 03:37:59 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:37:59 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:37:59 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:37:59 user nova-compute[72081]:   </os>
Apr 12 03:37:59 user nova-compute[72081]:   <features>
Apr 12 03:37:59 user nova-compute[72081]:     <acpi/>
Apr 12 03:37:59 user nova-compute[72081]:     <apic/>
Apr 12 03:37:59 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:37:59 user nova-compute[72081]:   </features>
Apr 12 03:37:59 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:37:59 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:37:59 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:37:59 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:37:59 user nova-compute[72081]:   </clock>
Apr 12 03:37:59 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:37:59 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:37:59 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:37:59 user nova-compute[72081]:   </cpu>
Apr 12 03:37:59 user nova-compute[72081]:   <devices>
Apr 12 03:37:59 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:37:59 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:37:59 user nova-compute[72081]:       <source dev="/dev/sde"/>
Apr 12 03:37:59 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:37:59 user nova-compute[72081]:       <serial>fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf</serial>
Apr 12 03:37:59 user nova-compute[72081]:     </disk>
Apr 12 03:37:59 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:37:59 user nova-compute[72081]:       <mac address="fa:16:3e:60:70:2a"/>
Apr 12 03:37:59 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:37:59 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:37:59 user nova-compute[72081]:       <target dev="tap50b78ba6-f0"/>
Apr 12 03:37:59 user nova-compute[72081]:     </interface>
Apr 12 03:37:59 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:37:59 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/1eef2c91-dfad-43aa-b193-25970db83b79/console.log" append="off"/>
Apr 12 03:37:59 user nova-compute[72081]:     </serial>
Apr 12 03:37:59 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:37:59 user nova-compute[72081]:     <video>
Apr 12 03:37:59 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:37:59 user nova-compute[72081]:     </video>
Apr 12 03:37:59 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:37:59 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:37:59 user nova-compute[72081]:     </rng>
Apr 12 03:37:59 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:37:59 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:37:59 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:37:59 user nova-compute[72081]:     </memballoon>
Apr 12 03:37:59 user nova-compute[72081]:   </devices>
Apr 12 03:37:59 user nova-compute[72081]: </domain>
Apr 12 03:37:59 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:37:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:37:52Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-volume-backed-server-966703655',display_name='tempest-TestVolumeBootPattern-volume-backed-server-966703655',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-volume-backed-server-966703655',id=21,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='003688353e2f4b36831c574ebaf65173',ramdisk_id='',reservation_id='r-ux4uytew',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-235043461',owner_user_name='tempest-TestVolumeBootPattern-235043461-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:37:54Z,user_data=None,user_id='b00b348933224558aad1cb12921f100f',uuid=1eef2c91-dfad-43aa-b193-25970db83b79,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "50b78ba6-f017-4643-a2ec-fa0f348bb270", "address": "fa:16:3e:60:70:2a", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50b78ba6-f0", "ovs_interfaceid": "50b78ba6-f017-4643-a2ec-fa0f348bb270", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:37:59 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converting VIF {"id": "50b78ba6-f017-4643-a2ec-fa0f348bb270", "address": "fa:16:3e:60:70:2a", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50b78ba6-f0", "ovs_interfaceid": "50b78ba6-f017-4643-a2ec-fa0f348bb270", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:37:59 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:60:70:2a,bridge_name='br-int',has_traffic_filtering=True,id=50b78ba6-f017-4643-a2ec-fa0f348bb270,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap50b78ba6-f0') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:37:59 user nova-compute[72081]: DEBUG os_vif [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:60:70:2a,bridge_name='br-int',has_traffic_filtering=True,id=50b78ba6-f017-4643-a2ec-fa0f348bb270,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap50b78ba6-f0') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:37:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:37:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:37:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap50b78ba6-f0, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:37:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap50b78ba6-f0, col_values=(('external_ids', {'iface-id': '50b78ba6-f017-4643-a2ec-fa0f348bb270', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:60:70:2a', 'vm-uuid': '1eef2c91-dfad-43aa-b193-25970db83b79'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:37:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:37:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:37:59 user nova-compute[72081]: INFO os_vif [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:60:70:2a,bridge_name='br-int',has_traffic_filtering=True,id=50b78ba6-f017-4643-a2ec-fa0f348bb270,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap50b78ba6-f0')
Apr 12 03:38:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:38:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] No VIF found with MAC fa:16:3e:60:70:2a, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:38:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:01 user nova-compute[72081]: DEBUG nova.compute.manager [req-b8d9b030-7afc-4b30-997d-393970f5bb59 req-95bb28c6-6c2b-4609-aec4-a331ba20b287 service nova] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Received event network-vif-plugged-50b78ba6-f017-4643-a2ec-fa0f348bb270 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:38:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b8d9b030-7afc-4b30-997d-393970f5bb59 req-95bb28c6-6c2b-4609-aec4-a331ba20b287 service nova] Acquiring lock "1eef2c91-dfad-43aa-b193-25970db83b79-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b8d9b030-7afc-4b30-997d-393970f5bb59 req-95bb28c6-6c2b-4609-aec4-a331ba20b287 service nova] Lock "1eef2c91-dfad-43aa-b193-25970db83b79-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b8d9b030-7afc-4b30-997d-393970f5bb59 req-95bb28c6-6c2b-4609-aec4-a331ba20b287 service nova] Lock "1eef2c91-dfad-43aa-b193-25970db83b79-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:01 user nova-compute[72081]: DEBUG nova.compute.manager [req-b8d9b030-7afc-4b30-997d-393970f5bb59 req-95bb28c6-6c2b-4609-aec4-a331ba20b287 service nova] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] No waiting events found dispatching network-vif-plugged-50b78ba6-f017-4643-a2ec-fa0f348bb270 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:38:01 user nova-compute[72081]: WARNING nova.compute.manager [req-b8d9b030-7afc-4b30-997d-393970f5bb59 req-95bb28c6-6c2b-4609-aec4-a331ba20b287 service nova] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Received unexpected event network-vif-plugged-50b78ba6-f017-4643-a2ec-fa0f348bb270 for instance with vm_state building and task_state spawning.
Apr 12 03:38:01 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:38:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:02 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:38:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:03 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270683.5875626, 1eef2c91-dfad-43aa-b193-25970db83b79 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:38:03 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] VM Resumed (Lifecycle Event)
Apr 12 03:38:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:38:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:38:03 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Instance spawned successfully.
Apr 12 03:38:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:38:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:38:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:38:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:03 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:38:03 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270683.5886753, 1eef2c91-dfad-43aa-b193-25970db83b79 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:38:03 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] VM Started (Lifecycle Event)
Apr 12 03:38:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-c609c787-f27f-46c0-9da6-8b6bca01be02 req-2ddb58a1-45a8-4f20-b024-b3f690e4a8e9 service nova] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Received event network-vif-plugged-50b78ba6-f017-4643-a2ec-fa0f348bb270 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:38:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c609c787-f27f-46c0-9da6-8b6bca01be02 req-2ddb58a1-45a8-4f20-b024-b3f690e4a8e9 service nova] Acquiring lock "1eef2c91-dfad-43aa-b193-25970db83b79-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c609c787-f27f-46c0-9da6-8b6bca01be02 req-2ddb58a1-45a8-4f20-b024-b3f690e4a8e9 service nova] Lock "1eef2c91-dfad-43aa-b193-25970db83b79-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c609c787-f27f-46c0-9da6-8b6bca01be02 req-2ddb58a1-45a8-4f20-b024-b3f690e4a8e9 service nova] Lock "1eef2c91-dfad-43aa-b193-25970db83b79-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-c609c787-f27f-46c0-9da6-8b6bca01be02 req-2ddb58a1-45a8-4f20-b024-b3f690e4a8e9 service nova] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] No waiting events found dispatching network-vif-plugged-50b78ba6-f017-4643-a2ec-fa0f348bb270 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:38:03 user nova-compute[72081]: WARNING nova.compute.manager [req-c609c787-f27f-46c0-9da6-8b6bca01be02 req-2ddb58a1-45a8-4f20-b024-b3f690e4a8e9 service nova] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Received unexpected event network-vif-plugged-50b78ba6-f017-4643-a2ec-fa0f348bb270 for instance with vm_state building and task_state spawning.
Apr 12 03:38:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:38:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:38:03 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:38:03 user nova-compute[72081]: INFO nova.compute.manager [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Took 5.25 seconds to spawn the instance on the hypervisor.
Apr 12 03:38:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:38:03 user nova-compute[72081]: INFO nova.compute.manager [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Took 10.45 seconds to build instance.
Apr 12 03:38:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ebbefb56-445c-40e7-8f07-6570f3679afe tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "1eef2c91-dfad-43aa-b193-25970db83b79" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 10.541s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:04 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:38:04 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:38:04 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:38:04 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:38:04 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:38:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:04 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:38:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.163s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.185s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdd (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:38:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json" returned: 0 in 0.159s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json" returned: 0 in 0.171s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:38:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquiring lock "ff5a7005-6dd4-491a-a55c-db7a82dc28f7" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "ff5a7005-6dd4-491a-a55c-db7a82dc28f7" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:05 user nova-compute[72081]: DEBUG nova.compute.manager [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:38:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:06 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:38:06 user nova-compute[72081]: INFO nova.compute.claims [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Claim successful on node user
Apr 12 03:38:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk --force-share --output=json" returned: 0 in 0.163s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk --force-share --output=json" returned: 0 in 0.166s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:38:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk --force-share --output=json" returned: 0 in 0.160s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:06 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:38:06 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:38:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk --force-share --output=json" returned: 0 in 0.160s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:38:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.645s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:06 user nova-compute[72081]: DEBUG nova.compute.manager [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:38:06 user nova-compute[72081]: DEBUG nova.compute.manager [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:38:06 user nova-compute[72081]: DEBUG nova.network.neutron [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:38:06 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:38:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json" returned: 0 in 0.188s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:06 user nova-compute[72081]: DEBUG nova.compute.manager [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0/disk --force-share --output=json" returned: 0 in 0.184s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:38:07 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Creating image(s)
Apr 12 03:38:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquiring lock "/opt/stack/data/nova/instances/ff5a7005-6dd4-491a-a55c-db7a82dc28f7/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "/opt/stack/data/nova/instances/ff5a7005-6dd4-491a-a55c-db7a82dc28f7/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.003s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "/opt/stack/data/nova/instances/ff5a7005-6dd4-491a-a55c-db7a82dc28f7/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.004s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG nova.policy [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'e2f496e5fe694d46a9803f797743a7dc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4cce206e2b94b03ade2d310373c5ca2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:38:07 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:38:07 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:38:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=7840MB free_disk=26.14081573486328GB free_vcpus=4 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.156s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 56834002-7593-4592-a86d-23bb90d00be0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 2d0904c3-74e0-47bb-a832-758a7035b2f5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance bcd267d2-5b2c-4ed2-b634-0210fae94dfd actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 48977eb2-ce43-4577-a1cb-340e35703dbd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 23367b24-0a3b-454d-901d-8a3e5396333a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 06481254-722a-4053-a6af-11f5dcc68636 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 1eef2c91-dfad-43aa-b193-25970db83b79 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance ff5a7005-6dd4-491a-a55c-db7a82dc28f7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=1664MB phys_disk=40GB used_disk=7GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.164s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/ff5a7005-6dd4-491a-a55c-db7a82dc28f7/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/ff5a7005-6dd4-491a-a55c-db7a82dc28f7/disk 1073741824" returned: 0 in 0.071s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.240s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.199s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:08 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Checking if we can resize image /opt/stack/data/nova/instances/ff5a7005-6dd4-491a-a55c-db7a82dc28f7/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:38:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ff5a7005-6dd4-491a-a55c-db7a82dc28f7/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:08 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:38:08 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:38:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ff5a7005-6dd4-491a-a55c-db7a82dc28f7/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:08 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Cannot resize image /opt/stack/data/nova/instances/ff5a7005-6dd4-491a-a55c-db7a82dc28f7/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:38:08 user nova-compute[72081]: DEBUG nova.objects.instance [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lazy-loading 'migration_context' on Instance uuid ff5a7005-6dd4-491a-a55c-db7a82dc28f7 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:38:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:38:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Ensure instance console log exists: /opt/stack/data/nova/instances/ff5a7005-6dd4-491a-a55c-db7a82dc28f7/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:38:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:38:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.681s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:09 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:38:09 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:38:09 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:38:09 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:38:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-bcd267d2-5b2c-4ed2-b634-0210fae94dfd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:38:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-bcd267d2-5b2c-4ed2-b634-0210fae94dfd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:38:09 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 03:38:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:09 user nova-compute[72081]: DEBUG nova.network.neutron [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Successfully created port: a460c7a2-ed31-445e-a2a8-296e5163084b {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:38:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:10 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Updating instance_info_cache with network_info: [{"id": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "address": "fa:16:3e:9d:0c:c2", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa4385731-05", "ovs_interfaceid": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:38:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-bcd267d2-5b2c-4ed2-b634-0210fae94dfd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:38:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 03:38:10 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:38:10 user nova-compute[72081]: DEBUG nova.network.neutron [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Successfully updated port: a460c7a2-ed31-445e-a2a8-296e5163084b {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:38:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquiring lock "refresh_cache-ff5a7005-6dd4-491a-a55c-db7a82dc28f7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:38:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquired lock "refresh_cache-ff5a7005-6dd4-491a-a55c-db7a82dc28f7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:38:10 user nova-compute[72081]: DEBUG nova.network.neutron [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:38:10 user nova-compute[72081]: DEBUG nova.compute.manager [req-2847e9e4-9470-48a7-a467-669bb069727d req-41d1f112-cfc3-4bb3-9dc2-f600af2a35e7 service nova] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Received event network-changed-a460c7a2-ed31-445e-a2a8-296e5163084b {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:38:10 user nova-compute[72081]: DEBUG nova.compute.manager [req-2847e9e4-9470-48a7-a467-669bb069727d req-41d1f112-cfc3-4bb3-9dc2-f600af2a35e7 service nova] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Refreshing instance network info cache due to event network-changed-a460c7a2-ed31-445e-a2a8-296e5163084b. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:38:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2847e9e4-9470-48a7-a467-669bb069727d req-41d1f112-cfc3-4bb3-9dc2-f600af2a35e7 service nova] Acquiring lock "refresh_cache-ff5a7005-6dd4-491a-a55c-db7a82dc28f7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.network.neutron [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.network.neutron [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Updating instance_info_cache with network_info: [{"id": "a460c7a2-ed31-445e-a2a8-296e5163084b", "address": "fa:16:3e:5a:90:ca", "network": {"id": "263f9508-8639-4289-9210-f9807acf6766", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-337713645-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f4cce206e2b94b03ade2d310373c5ca2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa460c7a2-ed", "ovs_interfaceid": "a460c7a2-ed31-445e-a2a8-296e5163084b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Releasing lock "refresh_cache-ff5a7005-6dd4-491a-a55c-db7a82dc28f7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.compute.manager [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Instance network_info: |[{"id": "a460c7a2-ed31-445e-a2a8-296e5163084b", "address": "fa:16:3e:5a:90:ca", "network": {"id": "263f9508-8639-4289-9210-f9807acf6766", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-337713645-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f4cce206e2b94b03ade2d310373c5ca2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa460c7a2-ed", "ovs_interfaceid": "a460c7a2-ed31-445e-a2a8-296e5163084b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2847e9e4-9470-48a7-a467-669bb069727d req-41d1f112-cfc3-4bb3-9dc2-f600af2a35e7 service nova] Acquired lock "refresh_cache-ff5a7005-6dd4-491a-a55c-db7a82dc28f7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.network.neutron [req-2847e9e4-9470-48a7-a467-669bb069727d req-41d1f112-cfc3-4bb3-9dc2-f600af2a35e7 service nova] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Refreshing network info cache for port a460c7a2-ed31-445e-a2a8-296e5163084b {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Start _get_guest_xml network_info=[{"id": "a460c7a2-ed31-445e-a2a8-296e5163084b", "address": "fa:16:3e:5a:90:ca", "network": {"id": "263f9508-8639-4289-9210-f9807acf6766", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-337713645-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f4cce206e2b94b03ade2d310373c5ca2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa460c7a2-ed", "ovs_interfaceid": "a460c7a2-ed31-445e-a2a8-296e5163084b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:38:11 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:38:11 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:38:05Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-1560370789',display_name='tempest-VolumesAdminNegativeTest-server-1560370789',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-1560370789',id=22,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f4cce206e2b94b03ade2d310373c5ca2',ramdisk_id='',reservation_id='r-h9pa9k9c',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesAdminNegativeTest-2105871059',owner_user_name='tempest-VolumesAdminNegativeTest-2105871059-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:38:07Z,user_data=None,user_id='e2f496e5fe694d46a9803f797743a7dc',uuid=ff5a7005-6dd4-491a-a55c-db7a82dc28f7,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a460c7a2-ed31-445e-a2a8-296e5163084b", "address": "fa:16:3e:5a:90:ca", "network": {"id": "263f9508-8639-4289-9210-f9807acf6766", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-337713645-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f4cce206e2b94b03ade2d310373c5ca2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa460c7a2-ed", "ovs_interfaceid": "a460c7a2-ed31-445e-a2a8-296e5163084b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Converting VIF {"id": "a460c7a2-ed31-445e-a2a8-296e5163084b", "address": "fa:16:3e:5a:90:ca", "network": {"id": "263f9508-8639-4289-9210-f9807acf6766", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-337713645-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f4cce206e2b94b03ade2d310373c5ca2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa460c7a2-ed", "ovs_interfaceid": "a460c7a2-ed31-445e-a2a8-296e5163084b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:5a:90:ca,bridge_name='br-int',has_traffic_filtering=True,id=a460c7a2-ed31-445e-a2a8-296e5163084b,network=Network(263f9508-8639-4289-9210-f9807acf6766),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa460c7a2-ed') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.objects.instance [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lazy-loading 'pci_devices' on Instance uuid ff5a7005-6dd4-491a-a55c-db7a82dc28f7 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:38:11 user nova-compute[72081]:   <uuid>ff5a7005-6dd4-491a-a55c-db7a82dc28f7</uuid>
Apr 12 03:38:11 user nova-compute[72081]:   <name>instance-00000016</name>
Apr 12 03:38:11 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:38:11 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:38:11 user nova-compute[72081]:   <metadata>
Apr 12 03:38:11 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:38:11 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:38:11 user nova-compute[72081]:       <nova:name>tempest-VolumesAdminNegativeTest-server-1560370789</nova:name>
Apr 12 03:38:11 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:38:11</nova:creationTime>
Apr 12 03:38:11 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:38:11 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:38:11 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:38:11 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:38:11 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:38:11 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:38:11 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:38:11 user nova-compute[72081]:       <nova:owner>
Apr 12 03:38:11 user nova-compute[72081]:         <nova:user uuid="e2f496e5fe694d46a9803f797743a7dc">tempest-VolumesAdminNegativeTest-2105871059-project-member</nova:user>
Apr 12 03:38:11 user nova-compute[72081]:         <nova:project uuid="f4cce206e2b94b03ade2d310373c5ca2">tempest-VolumesAdminNegativeTest-2105871059</nova:project>
Apr 12 03:38:11 user nova-compute[72081]:       </nova:owner>
Apr 12 03:38:11 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:38:11 user nova-compute[72081]:       <nova:ports>
Apr 12 03:38:11 user nova-compute[72081]:         <nova:port uuid="a460c7a2-ed31-445e-a2a8-296e5163084b">
Apr 12 03:38:11 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Apr 12 03:38:11 user nova-compute[72081]:         </nova:port>
Apr 12 03:38:11 user nova-compute[72081]:       </nova:ports>
Apr 12 03:38:11 user nova-compute[72081]:     </nova:instance>
Apr 12 03:38:11 user nova-compute[72081]:   </metadata>
Apr 12 03:38:11 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:38:11 user nova-compute[72081]:     <system>
Apr 12 03:38:11 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:38:11 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:38:11 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:38:11 user nova-compute[72081]:       <entry name="serial">ff5a7005-6dd4-491a-a55c-db7a82dc28f7</entry>
Apr 12 03:38:11 user nova-compute[72081]:       <entry name="uuid">ff5a7005-6dd4-491a-a55c-db7a82dc28f7</entry>
Apr 12 03:38:11 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:38:11 user nova-compute[72081]:     </system>
Apr 12 03:38:11 user nova-compute[72081]:   </sysinfo>
Apr 12 03:38:11 user nova-compute[72081]:   <os>
Apr 12 03:38:11 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:38:11 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:38:11 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:38:11 user nova-compute[72081]:   </os>
Apr 12 03:38:11 user nova-compute[72081]:   <features>
Apr 12 03:38:11 user nova-compute[72081]:     <acpi/>
Apr 12 03:38:11 user nova-compute[72081]:     <apic/>
Apr 12 03:38:11 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:38:11 user nova-compute[72081]:   </features>
Apr 12 03:38:11 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:38:11 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:38:11 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:38:11 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:38:11 user nova-compute[72081]:   </clock>
Apr 12 03:38:11 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:38:11 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:38:11 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:38:11 user nova-compute[72081]:   </cpu>
Apr 12 03:38:11 user nova-compute[72081]:   <devices>
Apr 12 03:38:11 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:38:11 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:38:11 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/ff5a7005-6dd4-491a-a55c-db7a82dc28f7/disk"/>
Apr 12 03:38:11 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:38:11 user nova-compute[72081]:     </disk>
Apr 12 03:38:11 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:38:11 user nova-compute[72081]:       <mac address="fa:16:3e:5a:90:ca"/>
Apr 12 03:38:11 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:38:11 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:38:11 user nova-compute[72081]:       <target dev="tapa460c7a2-ed"/>
Apr 12 03:38:11 user nova-compute[72081]:     </interface>
Apr 12 03:38:11 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:38:11 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/ff5a7005-6dd4-491a-a55c-db7a82dc28f7/console.log" append="off"/>
Apr 12 03:38:11 user nova-compute[72081]:     </serial>
Apr 12 03:38:11 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:38:11 user nova-compute[72081]:     <video>
Apr 12 03:38:11 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:38:11 user nova-compute[72081]:     </video>
Apr 12 03:38:11 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:38:11 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:38:11 user nova-compute[72081]:     </rng>
Apr 12 03:38:11 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:38:11 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:38:11 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:38:11 user nova-compute[72081]:     </memballoon>
Apr 12 03:38:11 user nova-compute[72081]:   </devices>
Apr 12 03:38:11 user nova-compute[72081]: </domain>
Apr 12 03:38:11 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:38:05Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-1560370789',display_name='tempest-VolumesAdminNegativeTest-server-1560370789',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-1560370789',id=22,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f4cce206e2b94b03ade2d310373c5ca2',ramdisk_id='',reservation_id='r-h9pa9k9c',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesAdminNegativeTest-2105871059',owner_user_name='tempest-VolumesAdminNegativeTest-2105871059-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:38:07Z,user_data=None,user_id='e2f496e5fe694d46a9803f797743a7dc',uuid=ff5a7005-6dd4-491a-a55c-db7a82dc28f7,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a460c7a2-ed31-445e-a2a8-296e5163084b", "address": "fa:16:3e:5a:90:ca", "network": {"id": "263f9508-8639-4289-9210-f9807acf6766", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-337713645-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f4cce206e2b94b03ade2d310373c5ca2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa460c7a2-ed", "ovs_interfaceid": "a460c7a2-ed31-445e-a2a8-296e5163084b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Converting VIF {"id": "a460c7a2-ed31-445e-a2a8-296e5163084b", "address": "fa:16:3e:5a:90:ca", "network": {"id": "263f9508-8639-4289-9210-f9807acf6766", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-337713645-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f4cce206e2b94b03ade2d310373c5ca2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa460c7a2-ed", "ovs_interfaceid": "a460c7a2-ed31-445e-a2a8-296e5163084b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:5a:90:ca,bridge_name='br-int',has_traffic_filtering=True,id=a460c7a2-ed31-445e-a2a8-296e5163084b,network=Network(263f9508-8639-4289-9210-f9807acf6766),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa460c7a2-ed') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG os_vif [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:5a:90:ca,bridge_name='br-int',has_traffic_filtering=True,id=a460c7a2-ed31-445e-a2a8-296e5163084b,network=Network(263f9508-8639-4289-9210-f9807acf6766),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa460c7a2-ed') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapa460c7a2-ed, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapa460c7a2-ed, col_values=(('external_ids', {'iface-id': 'a460c7a2-ed31-445e-a2a8-296e5163084b', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:5a:90:ca', 'vm-uuid': 'ff5a7005-6dd4-491a-a55c-db7a82dc28f7'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:11 user nova-compute[72081]: INFO os_vif [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:5a:90:ca,bridge_name='br-int',has_traffic_filtering=True,id=a460c7a2-ed31-445e-a2a8-296e5163084b,network=Network(263f9508-8639-4289-9210-f9807acf6766),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa460c7a2-ed')
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] No VIF found with MAC fa:16:3e:5a:90:ca, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.network.neutron [req-2847e9e4-9470-48a7-a467-669bb069727d req-41d1f112-cfc3-4bb3-9dc2-f600af2a35e7 service nova] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Updated VIF entry in instance network info cache for port a460c7a2-ed31-445e-a2a8-296e5163084b. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG nova.network.neutron [req-2847e9e4-9470-48a7-a467-669bb069727d req-41d1f112-cfc3-4bb3-9dc2-f600af2a35e7 service nova] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Updating instance_info_cache with network_info: [{"id": "a460c7a2-ed31-445e-a2a8-296e5163084b", "address": "fa:16:3e:5a:90:ca", "network": {"id": "263f9508-8639-4289-9210-f9807acf6766", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-337713645-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f4cce206e2b94b03ade2d310373c5ca2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa460c7a2-ed", "ovs_interfaceid": "a460c7a2-ed31-445e-a2a8-296e5163084b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:38:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2847e9e4-9470-48a7-a467-669bb069727d req-41d1f112-cfc3-4bb3-9dc2-f600af2a35e7 service nova] Releasing lock "refresh_cache-ff5a7005-6dd4-491a-a55c-db7a82dc28f7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:38:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:13 user nova-compute[72081]: DEBUG nova.compute.manager [req-c349e4b9-0efc-41ac-a388-fb14cf938f10 req-8e9a0408-5caf-497b-8b11-2f36c6c57241 service nova] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Received event network-vif-plugged-a460c7a2-ed31-445e-a2a8-296e5163084b {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:38:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c349e4b9-0efc-41ac-a388-fb14cf938f10 req-8e9a0408-5caf-497b-8b11-2f36c6c57241 service nova] Acquiring lock "ff5a7005-6dd4-491a-a55c-db7a82dc28f7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c349e4b9-0efc-41ac-a388-fb14cf938f10 req-8e9a0408-5caf-497b-8b11-2f36c6c57241 service nova] Lock "ff5a7005-6dd4-491a-a55c-db7a82dc28f7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c349e4b9-0efc-41ac-a388-fb14cf938f10 req-8e9a0408-5caf-497b-8b11-2f36c6c57241 service nova] Lock "ff5a7005-6dd4-491a-a55c-db7a82dc28f7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:13 user nova-compute[72081]: DEBUG nova.compute.manager [req-c349e4b9-0efc-41ac-a388-fb14cf938f10 req-8e9a0408-5caf-497b-8b11-2f36c6c57241 service nova] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] No waiting events found dispatching network-vif-plugged-a460c7a2-ed31-445e-a2a8-296e5163084b {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:38:13 user nova-compute[72081]: WARNING nova.compute.manager [req-c349e4b9-0efc-41ac-a388-fb14cf938f10 req-8e9a0408-5caf-497b-8b11-2f36c6c57241 service nova] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Received unexpected event network-vif-plugged-a460c7a2-ed31-445e-a2a8-296e5163084b for instance with vm_state building and task_state spawning.
Apr 12 03:38:15 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270695.0055501, ff5a7005-6dd4-491a-a55c-db7a82dc28f7 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:38:15 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] VM Resumed (Lifecycle Event)
Apr 12 03:38:15 user nova-compute[72081]: DEBUG nova.compute.manager [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:38:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:38:15 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Instance spawned successfully.
Apr 12 03:38:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:38:15 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:38:15 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:38:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:15 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:38:15 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270695.0058398, ff5a7005-6dd4-491a-a55c-db7a82dc28f7 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:38:15 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] VM Started (Lifecycle Event)
Apr 12 03:38:15 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:38:15 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:38:15 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:38:15 user nova-compute[72081]: INFO nova.compute.manager [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Took 8.06 seconds to spawn the instance on the hypervisor.
Apr 12 03:38:15 user nova-compute[72081]: DEBUG nova.compute.manager [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:38:15 user nova-compute[72081]: DEBUG nova.compute.manager [req-79170971-b9f1-49cf-a05d-2dfffd004e13 req-2a9fa1aa-ed49-4f2c-adfa-316235bbdbaf service nova] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Received event network-vif-plugged-a460c7a2-ed31-445e-a2a8-296e5163084b {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:38:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-79170971-b9f1-49cf-a05d-2dfffd004e13 req-2a9fa1aa-ed49-4f2c-adfa-316235bbdbaf service nova] Acquiring lock "ff5a7005-6dd4-491a-a55c-db7a82dc28f7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-79170971-b9f1-49cf-a05d-2dfffd004e13 req-2a9fa1aa-ed49-4f2c-adfa-316235bbdbaf service nova] Lock "ff5a7005-6dd4-491a-a55c-db7a82dc28f7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-79170971-b9f1-49cf-a05d-2dfffd004e13 req-2a9fa1aa-ed49-4f2c-adfa-316235bbdbaf service nova] Lock "ff5a7005-6dd4-491a-a55c-db7a82dc28f7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:15 user nova-compute[72081]: DEBUG nova.compute.manager [req-79170971-b9f1-49cf-a05d-2dfffd004e13 req-2a9fa1aa-ed49-4f2c-adfa-316235bbdbaf service nova] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] No waiting events found dispatching network-vif-plugged-a460c7a2-ed31-445e-a2a8-296e5163084b {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:38:15 user nova-compute[72081]: WARNING nova.compute.manager [req-79170971-b9f1-49cf-a05d-2dfffd004e13 req-2a9fa1aa-ed49-4f2c-adfa-316235bbdbaf service nova] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Received unexpected event network-vif-plugged-a460c7a2-ed31-445e-a2a8-296e5163084b for instance with vm_state building and task_state spawning.
Apr 12 03:38:15 user nova-compute[72081]: INFO nova.compute.manager [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Took 9.22 seconds to build instance.
Apr 12 03:38:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-44195801-72c7-47f1-b2b5-ca33678dd4d7 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "ff5a7005-6dd4-491a-a55c-db7a82dc28f7" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 9.330s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:16 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:29 user nova-compute[72081]: DEBUG nova.compute.manager [req-c2d2a0ca-fa4b-4964-ba6d-0050ced191ab req-dbd0ace7-f09f-4b00-bdab-68181306fde7 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Received event network-changed-a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:38:29 user nova-compute[72081]: DEBUG nova.compute.manager [req-c2d2a0ca-fa4b-4964-ba6d-0050ced191ab req-dbd0ace7-f09f-4b00-bdab-68181306fde7 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Refreshing instance network info cache due to event network-changed-a76c6f07-234b-4832-8ace-ec81656b06a7. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:38:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c2d2a0ca-fa4b-4964-ba6d-0050ced191ab req-dbd0ace7-f09f-4b00-bdab-68181306fde7 service nova] Acquiring lock "refresh_cache-23367b24-0a3b-454d-901d-8a3e5396333a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:38:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c2d2a0ca-fa4b-4964-ba6d-0050ced191ab req-dbd0ace7-f09f-4b00-bdab-68181306fde7 service nova] Acquired lock "refresh_cache-23367b24-0a3b-454d-901d-8a3e5396333a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:38:29 user nova-compute[72081]: DEBUG nova.network.neutron [req-c2d2a0ca-fa4b-4964-ba6d-0050ced191ab req-dbd0ace7-f09f-4b00-bdab-68181306fde7 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Refreshing network info cache for port a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:38:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-cf757f78-d0ec-49a4-b5a7-9afc20f442ad req-71fdfc01-854f-4faa-be8e-a19646583634 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Received event network-changed-a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:38:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-cf757f78-d0ec-49a4-b5a7-9afc20f442ad req-71fdfc01-854f-4faa-be8e-a19646583634 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Refreshing instance network info cache due to event network-changed-a76c6f07-234b-4832-8ace-ec81656b06a7. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:38:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cf757f78-d0ec-49a4-b5a7-9afc20f442ad req-71fdfc01-854f-4faa-be8e-a19646583634 service nova] Acquiring lock "refresh_cache-23367b24-0a3b-454d-901d-8a3e5396333a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:38:30 user nova-compute[72081]: DEBUG nova.network.neutron [req-c2d2a0ca-fa4b-4964-ba6d-0050ced191ab req-dbd0ace7-f09f-4b00-bdab-68181306fde7 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Updated VIF entry in instance network info cache for port a76c6f07-234b-4832-8ace-ec81656b06a7. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:38:30 user nova-compute[72081]: DEBUG nova.network.neutron [req-c2d2a0ca-fa4b-4964-ba6d-0050ced191ab req-dbd0ace7-f09f-4b00-bdab-68181306fde7 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Updating instance_info_cache with network_info: [{"id": "a76c6f07-234b-4832-8ace-ec81656b06a7", "address": "fa:16:3e:e4:18:3e", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.29", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa76c6f07-23", "ovs_interfaceid": "a76c6f07-234b-4832-8ace-ec81656b06a7", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:38:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c2d2a0ca-fa4b-4964-ba6d-0050ced191ab req-dbd0ace7-f09f-4b00-bdab-68181306fde7 service nova] Releasing lock "refresh_cache-23367b24-0a3b-454d-901d-8a3e5396333a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:38:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cf757f78-d0ec-49a4-b5a7-9afc20f442ad req-71fdfc01-854f-4faa-be8e-a19646583634 service nova] Acquired lock "refresh_cache-23367b24-0a3b-454d-901d-8a3e5396333a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:38:30 user nova-compute[72081]: DEBUG nova.network.neutron [req-cf757f78-d0ec-49a4-b5a7-9afc20f442ad req-71fdfc01-854f-4faa-be8e-a19646583634 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Refreshing network info cache for port a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:38:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "23367b24-0a3b-454d-901d-8a3e5396333a" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "23367b24-0a3b-454d-901d-8a3e5396333a" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:30 user nova-compute[72081]: INFO nova.compute.manager [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Detaching volume 7e52a442-8ea4-45e3-98eb-dc605d0d1d55
Apr 12 03:38:30 user nova-compute[72081]: INFO nova.virt.block_device [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Attempting to driver detach volume 7e52a442-8ea4-45e3-98eb-dc605d0d1d55 from mountpoint /dev/vdb
Apr 12 03:38:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Attempting to detach device vdb from instance 23367b24-0a3b-454d-901d-8a3e5396333a from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 03:38:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:38:30 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:38:30 user nova-compute[72081]:   <source dev="/dev/sdc"/>
Apr 12 03:38:30 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:38:30 user nova-compute[72081]:   <serial>7e52a442-8ea4-45e3-98eb-dc605d0d1d55</serial>
Apr 12 03:38:30 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Apr 12 03:38:30 user nova-compute[72081]: </disk>
Apr 12 03:38:30 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:38:30 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Successfully detached device vdb from instance 23367b24-0a3b-454d-901d-8a3e5396333a from the persistent domain config.
Apr 12 03:38:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 23367b24-0a3b-454d-901d-8a3e5396333a from the live domain config. {{(pid=72081) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 12 03:38:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:38:30 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:38:30 user nova-compute[72081]:   <source dev="/dev/sdc"/>
Apr 12 03:38:30 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:38:30 user nova-compute[72081]:   <serial>7e52a442-8ea4-45e3-98eb-dc605d0d1d55</serial>
Apr 12 03:38:30 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Apr 12 03:38:30 user nova-compute[72081]: </disk>
Apr 12 03:38:30 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG nova.network.neutron [req-cf757f78-d0ec-49a4-b5a7-9afc20f442ad req-71fdfc01-854f-4faa-be8e-a19646583634 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Updated VIF entry in instance network info cache for port a76c6f07-234b-4832-8ace-ec81656b06a7. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG nova.network.neutron [req-cf757f78-d0ec-49a4-b5a7-9afc20f442ad req-71fdfc01-854f-4faa-be8e-a19646583634 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Updating instance_info_cache with network_info: [{"id": "a76c6f07-234b-4832-8ace-ec81656b06a7", "address": "fa:16:3e:e4:18:3e", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa76c6f07-23", "ovs_interfaceid": "a76c6f07-234b-4832-8ace-ec81656b06a7", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cf757f78-d0ec-49a4-b5a7-9afc20f442ad req-71fdfc01-854f-4faa-be8e-a19646583634 service nova] Releasing lock "refresh_cache-23367b24-0a3b-454d-901d-8a3e5396333a" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 23367b24-0a3b-454d-901d-8a3e5396333a {{(pid=72081) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Received event <DeviceRemovedEvent: 1681270711.4639673, 23367b24-0a3b-454d-901d-8a3e5396333a => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72081) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 12 03:38:31 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Successfully detached device vdb from instance 23367b24-0a3b-454d-901d-8a3e5396333a from the live domain config.
Apr 12 03:38:31 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'yUzCJkbp', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:38:31 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:38:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[28372944-735b-41f8-852d-fcb9f165a16f]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:38:31 user nova-compute[72081]: No targets found.
Apr 12 03:38:31 user nova-compute[72081]: iSNS:
Apr 12 03:38:31 user nova-compute[72081]: No targets found.
Apr 12 03:38:31 user nova-compute[72081]: STATIC:
Apr 12 03:38:31 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf
Apr 12 03:38:31 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:38:31 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:38:31 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:38:31 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:38:31 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:38:31 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a
Apr 12 03:38:31 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:38:31 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:38:31 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55
Apr 12 03:38:31 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:38:31 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:38:31 user nova-compute[72081]: FIRMWARE:
Apr 12 03:38:31 user nova-compute[72081]: No targets found.
Apr 12 03:38:31 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:38:31 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:38:31 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ea5dc62b-3ab7-4415-be65-d02230659a86]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.031s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d39c69d2-f8ad-4a90-babe-40be6105b7d3]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:38:31 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:38:31 user nova-compute[72081]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)
Apr 12 03:38:31 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:38:31 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:38:31 user nova-compute[72081]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 (non-flash)
Apr 12 03:38:31 user nova-compute[72081]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)
Apr 12 03:38:31 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:38:31 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fba3b4d30>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55'): ({'sdc'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Removing single pathed devices sdc {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe7cf787-aab3-4eae-9f23-3a03d4de3168]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e6f00855-ae26-48ae-9bde-8b83c8982124]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Flushing IO for device /dev/sdc {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.013s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[03e7e6a9-7e08-49aa-976f-cbac718cd0c7]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.050s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[37e137d9-e8c5-43b4-860d-50383769f0ca]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] SCSI volumes sdc have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c25483c9-abea-4dd7-840b-a47102083a7c]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --logout" returned: 0 in 0.063s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[cf21825a-3666-4469-9351-b307b3d643af]: (4, ('Logging out of session [sid: 21, target: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55, portal: 172.16.0.220,3260]\nLogout of [sid: 21, target: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 21, target: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55, portal: 172.16.0.220,3260]
Apr 12 03:38:31 user nova-compute[72081]: Logout of [sid: 21, target: iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55, portal: 172.16.0.220,3260] successful.
Apr 12 03:38:31 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e52a442-8ea4-45e3-98eb-dc605d0d1d55 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.028s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3b2dd0c5-a97a-44a4-93e3-43b33cc27613]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.341s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] <== disconnect_volume: return (344ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:38:31 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:38:32 user nova-compute[72081]: DEBUG nova.objects.instance [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lazy-loading 'flavor' on Instance uuid 23367b24-0a3b-454d-901d-8a3e5396333a {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:38:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d9a9c876-1619-4c33-bb49-f5670e700edf tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "23367b24-0a3b-454d-901d-8a3e5396333a" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.773s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:34 user nova-compute[72081]: DEBUG nova.compute.manager [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:38:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:38:34 user nova-compute[72081]: INFO nova.compute.claims [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Claim successful on node user
Apr 12 03:38:34 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:38:34 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.407s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG nova.compute.manager [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG nova.compute.manager [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG nova.network.neutron [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:38:35 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:38:35 user nova-compute[72081]: DEBUG nova.compute.manager [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG nova.policy [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd52637cca2114e5e99e688ee371c835a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd13546149a4e44e1b782b99e9eb452e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG nova.compute.manager [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:38:35 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Creating image(s)
Apr 12 03:38:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "/opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "/opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "/opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.004s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.150s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk 1073741824" returned: 0 in 0.053s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.197s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.153s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Checking if we can resize image /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG nova.network.neutron [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Successfully created port: 979f4007-c0b3-408e-9f1c-a62c05af6417 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json" returned: 0 in 0.150s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Cannot resize image /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG nova.objects.instance [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lazy-loading 'migration_context' on Instance uuid f73fa883-75f0-4593-a89c-d009c6d4ebf0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Ensure instance console log exists: /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.network.neutron [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Successfully updated port: 979f4007-c0b3-408e-9f1c-a62c05af6417 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "refresh_cache-f73fa883-75f0-4593-a89c-d009c6d4ebf0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquired lock "refresh_cache-f73fa883-75f0-4593-a89c-d009c6d4ebf0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.network.neutron [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.compute.manager [req-2654fcf2-719d-4247-9d74-81c99c25c708 req-2d49d7c6-f9fa-46aa-b3f5-f774571b803b service nova] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Received event network-changed-979f4007-c0b3-408e-9f1c-a62c05af6417 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.compute.manager [req-2654fcf2-719d-4247-9d74-81c99c25c708 req-2d49d7c6-f9fa-46aa-b3f5-f774571b803b service nova] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Refreshing instance network info cache due to event network-changed-979f4007-c0b3-408e-9f1c-a62c05af6417. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2654fcf2-719d-4247-9d74-81c99c25c708 req-2d49d7c6-f9fa-46aa-b3f5-f774571b803b service nova] Acquiring lock "refresh_cache-f73fa883-75f0-4593-a89c-d009c6d4ebf0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.network.neutron [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.network.neutron [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Updating instance_info_cache with network_info: [{"id": "979f4007-c0b3-408e-9f1c-a62c05af6417", "address": "fa:16:3e:fd:fc:16", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap979f4007-c0", "ovs_interfaceid": "979f4007-c0b3-408e-9f1c-a62c05af6417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Releasing lock "refresh_cache-f73fa883-75f0-4593-a89c-d009c6d4ebf0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.compute.manager [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Instance network_info: |[{"id": "979f4007-c0b3-408e-9f1c-a62c05af6417", "address": "fa:16:3e:fd:fc:16", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap979f4007-c0", "ovs_interfaceid": "979f4007-c0b3-408e-9f1c-a62c05af6417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2654fcf2-719d-4247-9d74-81c99c25c708 req-2d49d7c6-f9fa-46aa-b3f5-f774571b803b service nova] Acquired lock "refresh_cache-f73fa883-75f0-4593-a89c-d009c6d4ebf0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.network.neutron [req-2654fcf2-719d-4247-9d74-81c99c25c708 req-2d49d7c6-f9fa-46aa-b3f5-f774571b803b service nova] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Refreshing network info cache for port 979f4007-c0b3-408e-9f1c-a62c05af6417 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Start _get_guest_xml network_info=[{"id": "979f4007-c0b3-408e-9f1c-a62c05af6417", "address": "fa:16:3e:fd:fc:16", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap979f4007-c0", "ovs_interfaceid": "979f4007-c0b3-408e-9f1c-a62c05af6417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:38:36 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:38:36 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:38:34Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-1062487719',display_name='tempest-AttachVolumeNegativeTest-server-1062487719',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-1062487719',id=23,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS13JiPBnpwy6hWDJrnaZHwuwdL5vH5nyF6/MTnPZMJQcLak2kXYJ42hH5AmlQraI/A/u9o/w1zAFpUYDoJwJLsZ7venHMIO1i9s9kjpJA5JVZMg0XUxqXqwfxeNoHNZQ==',key_name='tempest-keypair-1411992271',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='d13546149a4e44e1b782b99e9eb452e8',ramdisk_id='',reservation_id='r-5zkadp10',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-494105310',owner_user_name='tempest-AttachVolumeNegativeTest-494105310-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:38:35Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d52637cca2114e5e99e688ee371c835a',uuid=f73fa883-75f0-4593-a89c-d009c6d4ebf0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "979f4007-c0b3-408e-9f1c-a62c05af6417", "address": "fa:16:3e:fd:fc:16", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap979f4007-c0", "ovs_interfaceid": "979f4007-c0b3-408e-9f1c-a62c05af6417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Converting VIF {"id": "979f4007-c0b3-408e-9f1c-a62c05af6417", "address": "fa:16:3e:fd:fc:16", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap979f4007-c0", "ovs_interfaceid": "979f4007-c0b3-408e-9f1c-a62c05af6417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:fd:fc:16,bridge_name='br-int',has_traffic_filtering=True,id=979f4007-c0b3-408e-9f1c-a62c05af6417,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap979f4007-c0') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.objects.instance [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lazy-loading 'pci_devices' on Instance uuid f73fa883-75f0-4593-a89c-d009c6d4ebf0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:38:36 user nova-compute[72081]:   <uuid>f73fa883-75f0-4593-a89c-d009c6d4ebf0</uuid>
Apr 12 03:38:36 user nova-compute[72081]:   <name>instance-00000017</name>
Apr 12 03:38:36 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:38:36 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:38:36 user nova-compute[72081]:   <metadata>
Apr 12 03:38:36 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:38:36 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:38:36 user nova-compute[72081]:       <nova:name>tempest-AttachVolumeNegativeTest-server-1062487719</nova:name>
Apr 12 03:38:36 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:38:36</nova:creationTime>
Apr 12 03:38:36 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:38:36 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:38:36 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:38:36 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:38:36 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:38:36 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:38:36 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:38:36 user nova-compute[72081]:       <nova:owner>
Apr 12 03:38:36 user nova-compute[72081]:         <nova:user uuid="d52637cca2114e5e99e688ee371c835a">tempest-AttachVolumeNegativeTest-494105310-project-member</nova:user>
Apr 12 03:38:36 user nova-compute[72081]:         <nova:project uuid="d13546149a4e44e1b782b99e9eb452e8">tempest-AttachVolumeNegativeTest-494105310</nova:project>
Apr 12 03:38:36 user nova-compute[72081]:       </nova:owner>
Apr 12 03:38:36 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:38:36 user nova-compute[72081]:       <nova:ports>
Apr 12 03:38:36 user nova-compute[72081]:         <nova:port uuid="979f4007-c0b3-408e-9f1c-a62c05af6417">
Apr 12 03:38:36 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Apr 12 03:38:36 user nova-compute[72081]:         </nova:port>
Apr 12 03:38:36 user nova-compute[72081]:       </nova:ports>
Apr 12 03:38:36 user nova-compute[72081]:     </nova:instance>
Apr 12 03:38:36 user nova-compute[72081]:   </metadata>
Apr 12 03:38:36 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:38:36 user nova-compute[72081]:     <system>
Apr 12 03:38:36 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:38:36 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:38:36 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:38:36 user nova-compute[72081]:       <entry name="serial">f73fa883-75f0-4593-a89c-d009c6d4ebf0</entry>
Apr 12 03:38:36 user nova-compute[72081]:       <entry name="uuid">f73fa883-75f0-4593-a89c-d009c6d4ebf0</entry>
Apr 12 03:38:36 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:38:36 user nova-compute[72081]:     </system>
Apr 12 03:38:36 user nova-compute[72081]:   </sysinfo>
Apr 12 03:38:36 user nova-compute[72081]:   <os>
Apr 12 03:38:36 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:38:36 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:38:36 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:38:36 user nova-compute[72081]:   </os>
Apr 12 03:38:36 user nova-compute[72081]:   <features>
Apr 12 03:38:36 user nova-compute[72081]:     <acpi/>
Apr 12 03:38:36 user nova-compute[72081]:     <apic/>
Apr 12 03:38:36 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:38:36 user nova-compute[72081]:   </features>
Apr 12 03:38:36 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:38:36 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:38:36 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:38:36 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:38:36 user nova-compute[72081]:   </clock>
Apr 12 03:38:36 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:38:36 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:38:36 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:38:36 user nova-compute[72081]:   </cpu>
Apr 12 03:38:36 user nova-compute[72081]:   <devices>
Apr 12 03:38:36 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:38:36 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:38:36 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk"/>
Apr 12 03:38:36 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:38:36 user nova-compute[72081]:     </disk>
Apr 12 03:38:36 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:38:36 user nova-compute[72081]:       <mac address="fa:16:3e:fd:fc:16"/>
Apr 12 03:38:36 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:38:36 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:38:36 user nova-compute[72081]:       <target dev="tap979f4007-c0"/>
Apr 12 03:38:36 user nova-compute[72081]:     </interface>
Apr 12 03:38:36 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:38:36 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/console.log" append="off"/>
Apr 12 03:38:36 user nova-compute[72081]:     </serial>
Apr 12 03:38:36 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:38:36 user nova-compute[72081]:     <video>
Apr 12 03:38:36 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:38:36 user nova-compute[72081]:     </video>
Apr 12 03:38:36 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:38:36 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:38:36 user nova-compute[72081]:     </rng>
Apr 12 03:38:36 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:38:36 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:38:36 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:38:36 user nova-compute[72081]:     </memballoon>
Apr 12 03:38:36 user nova-compute[72081]:   </devices>
Apr 12 03:38:36 user nova-compute[72081]: </domain>
Apr 12 03:38:36 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:38:34Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-1062487719',display_name='tempest-AttachVolumeNegativeTest-server-1062487719',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-1062487719',id=23,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS13JiPBnpwy6hWDJrnaZHwuwdL5vH5nyF6/MTnPZMJQcLak2kXYJ42hH5AmlQraI/A/u9o/w1zAFpUYDoJwJLsZ7venHMIO1i9s9kjpJA5JVZMg0XUxqXqwfxeNoHNZQ==',key_name='tempest-keypair-1411992271',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='d13546149a4e44e1b782b99e9eb452e8',ramdisk_id='',reservation_id='r-5zkadp10',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-494105310',owner_user_name='tempest-AttachVolumeNegativeTest-494105310-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:38:35Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d52637cca2114e5e99e688ee371c835a',uuid=f73fa883-75f0-4593-a89c-d009c6d4ebf0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "979f4007-c0b3-408e-9f1c-a62c05af6417", "address": "fa:16:3e:fd:fc:16", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap979f4007-c0", "ovs_interfaceid": "979f4007-c0b3-408e-9f1c-a62c05af6417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Converting VIF {"id": "979f4007-c0b3-408e-9f1c-a62c05af6417", "address": "fa:16:3e:fd:fc:16", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap979f4007-c0", "ovs_interfaceid": "979f4007-c0b3-408e-9f1c-a62c05af6417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:fd:fc:16,bridge_name='br-int',has_traffic_filtering=True,id=979f4007-c0b3-408e-9f1c-a62c05af6417,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap979f4007-c0') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG os_vif [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:fd:fc:16,bridge_name='br-int',has_traffic_filtering=True,id=979f4007-c0b3-408e-9f1c-a62c05af6417,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap979f4007-c0') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap979f4007-c0, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:38:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap979f4007-c0, col_values=(('external_ids', {'iface-id': '979f4007-c0b3-408e-9f1c-a62c05af6417', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:fd:fc:16', 'vm-uuid': 'f73fa883-75f0-4593-a89c-d009c6d4ebf0'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:38:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:38:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:37 user nova-compute[72081]: INFO os_vif [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:fd:fc:16,bridge_name='br-int',has_traffic_filtering=True,id=979f4007-c0b3-408e-9f1c-a62c05af6417,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap979f4007-c0')
Apr 12 03:38:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:38:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] No VIF found with MAC fa:16:3e:fd:fc:16, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:38:37 user nova-compute[72081]: DEBUG nova.network.neutron [req-2654fcf2-719d-4247-9d74-81c99c25c708 req-2d49d7c6-f9fa-46aa-b3f5-f774571b803b service nova] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Updated VIF entry in instance network info cache for port 979f4007-c0b3-408e-9f1c-a62c05af6417. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:38:37 user nova-compute[72081]: DEBUG nova.network.neutron [req-2654fcf2-719d-4247-9d74-81c99c25c708 req-2d49d7c6-f9fa-46aa-b3f5-f774571b803b service nova] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Updating instance_info_cache with network_info: [{"id": "979f4007-c0b3-408e-9f1c-a62c05af6417", "address": "fa:16:3e:fd:fc:16", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap979f4007-c0", "ovs_interfaceid": "979f4007-c0b3-408e-9f1c-a62c05af6417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:38:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2654fcf2-719d-4247-9d74-81c99c25c708 req-2d49d7c6-f9fa-46aa-b3f5-f774571b803b service nova] Releasing lock "refresh_cache-f73fa883-75f0-4593-a89c-d009c6d4ebf0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:38:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-c721772c-ff6b-47bc-9151-5034453422c5 req-3d20544a-ae76-4a92-ab66-83db768caa5a service nova] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Received event network-vif-plugged-979f4007-c0b3-408e-9f1c-a62c05af6417 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:38:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c721772c-ff6b-47bc-9151-5034453422c5 req-3d20544a-ae76-4a92-ab66-83db768caa5a service nova] Acquiring lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c721772c-ff6b-47bc-9151-5034453422c5 req-3d20544a-ae76-4a92-ab66-83db768caa5a service nova] Lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c721772c-ff6b-47bc-9151-5034453422c5 req-3d20544a-ae76-4a92-ab66-83db768caa5a service nova] Lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-c721772c-ff6b-47bc-9151-5034453422c5 req-3d20544a-ae76-4a92-ab66-83db768caa5a service nova] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] No waiting events found dispatching network-vif-plugged-979f4007-c0b3-408e-9f1c-a62c05af6417 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:38:38 user nova-compute[72081]: WARNING nova.compute.manager [req-c721772c-ff6b-47bc-9151-5034453422c5 req-3d20544a-ae76-4a92-ab66-83db768caa5a service nova] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Received unexpected event network-vif-plugged-979f4007-c0b3-408e-9f1c-a62c05af6417 for instance with vm_state building and task_state spawning.
Apr 12 03:38:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:40 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270720.574508, f73fa883-75f0-4593-a89c-d009c6d4ebf0 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:38:40 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] VM Resumed (Lifecycle Event)
Apr 12 03:38:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:38:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:38:40 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Instance spawned successfully.
Apr 12 03:38:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:38:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:38:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:38:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:40 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:38:40 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270720.5778139, f73fa883-75f0-4593-a89c-d009c6d4ebf0 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:38:40 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] VM Started (Lifecycle Event)
Apr 12 03:38:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:38:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:38:40 user nova-compute[72081]: INFO nova.compute.manager [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Took 5.49 seconds to spawn the instance on the hypervisor.
Apr 12 03:38:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:38:40 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:38:40 user nova-compute[72081]: DEBUG nova.compute.manager [req-41f9d095-fb92-4ae4-840e-436b1e2b1e67 req-1627c191-ffc2-42d8-8deb-e858fcad72ac service nova] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Received event network-vif-plugged-979f4007-c0b3-408e-9f1c-a62c05af6417 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:38:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-41f9d095-fb92-4ae4-840e-436b1e2b1e67 req-1627c191-ffc2-42d8-8deb-e858fcad72ac service nova] Acquiring lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-41f9d095-fb92-4ae4-840e-436b1e2b1e67 req-1627c191-ffc2-42d8-8deb-e858fcad72ac service nova] Lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-41f9d095-fb92-4ae4-840e-436b1e2b1e67 req-1627c191-ffc2-42d8-8deb-e858fcad72ac service nova] Lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:40 user nova-compute[72081]: DEBUG nova.compute.manager [req-41f9d095-fb92-4ae4-840e-436b1e2b1e67 req-1627c191-ffc2-42d8-8deb-e858fcad72ac service nova] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] No waiting events found dispatching network-vif-plugged-979f4007-c0b3-408e-9f1c-a62c05af6417 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:38:40 user nova-compute[72081]: WARNING nova.compute.manager [req-41f9d095-fb92-4ae4-840e-436b1e2b1e67 req-1627c191-ffc2-42d8-8deb-e858fcad72ac service nova] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Received unexpected event network-vif-plugged-979f4007-c0b3-408e-9f1c-a62c05af6417 for instance with vm_state building and task_state spawning.
Apr 12 03:38:40 user nova-compute[72081]: INFO nova.compute.manager [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Took 6.24 seconds to build instance.
Apr 12 03:38:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-82612336-b62e-4dbc-8265-398d14d2f6c7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.335s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "3d4407f4-e078-4777-a110-cb3f922be2e8" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "3d4407f4-e078-4777-a110-cb3f922be2e8" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:43 user nova-compute[72081]: DEBUG nova.compute.manager [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:38:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:43 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:38:43 user nova-compute[72081]: INFO nova.compute.claims [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Claim successful on node user
Apr 12 03:38:43 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:38:43 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:38:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.464s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:43 user nova-compute[72081]: DEBUG nova.compute.manager [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:38:43 user nova-compute[72081]: DEBUG nova.compute.manager [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:38:43 user nova-compute[72081]: DEBUG nova.network.neutron [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:38:43 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:38:43 user nova-compute[72081]: DEBUG nova.compute.manager [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG nova.policy [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd99be569057f432484e4a60b46882bbd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e1fa352a8c144a1aada7ce0d8818429', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:38:44 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Creating image(s)
Apr 12 03:38:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "/opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "/opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "/opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.004s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.170s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.003s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk 1073741824" returned: 0 in 0.059s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.206s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.157s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Checking if we can resize image /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG nova.network.neutron [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Successfully created port: aec6d617-eb2b-42b3-8cce-f21de7c1b91c {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json" returned: 0 in 0.151s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Cannot resize image /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG nova.objects.instance [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lazy-loading 'migration_context' on Instance uuid 3d4407f4-e078-4777-a110-cb3f922be2e8 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Ensure instance console log exists: /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:45 user nova-compute[72081]: DEBUG nova.network.neutron [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Successfully updated port: aec6d617-eb2b-42b3-8cce-f21de7c1b91c {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:38:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "refresh_cache-3d4407f4-e078-4777-a110-cb3f922be2e8" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:38:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquired lock "refresh_cache-3d4407f4-e078-4777-a110-cb3f922be2e8" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:38:45 user nova-compute[72081]: DEBUG nova.network.neutron [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:38:45 user nova-compute[72081]: DEBUG nova.compute.manager [req-71fdf42e-afaf-4c55-95fb-ef7a4d985a36 req-2df015a8-2e36-4b3a-98e4-d160ce0d8aa4 service nova] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Received event network-changed-aec6d617-eb2b-42b3-8cce-f21de7c1b91c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:38:45 user nova-compute[72081]: DEBUG nova.compute.manager [req-71fdf42e-afaf-4c55-95fb-ef7a4d985a36 req-2df015a8-2e36-4b3a-98e4-d160ce0d8aa4 service nova] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Refreshing instance network info cache due to event network-changed-aec6d617-eb2b-42b3-8cce-f21de7c1b91c. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:38:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-71fdf42e-afaf-4c55-95fb-ef7a4d985a36 req-2df015a8-2e36-4b3a-98e4-d160ce0d8aa4 service nova] Acquiring lock "refresh_cache-3d4407f4-e078-4777-a110-cb3f922be2e8" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:38:45 user nova-compute[72081]: DEBUG nova.network.neutron [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.network.neutron [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Updating instance_info_cache with network_info: [{"id": "aec6d617-eb2b-42b3-8cce-f21de7c1b91c", "address": "fa:16:3e:92:a6:05", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapaec6d617-eb", "ovs_interfaceid": "aec6d617-eb2b-42b3-8cce-f21de7c1b91c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Releasing lock "refresh_cache-3d4407f4-e078-4777-a110-cb3f922be2e8" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.compute.manager [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Instance network_info: |[{"id": "aec6d617-eb2b-42b3-8cce-f21de7c1b91c", "address": "fa:16:3e:92:a6:05", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapaec6d617-eb", "ovs_interfaceid": "aec6d617-eb2b-42b3-8cce-f21de7c1b91c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-71fdf42e-afaf-4c55-95fb-ef7a4d985a36 req-2df015a8-2e36-4b3a-98e4-d160ce0d8aa4 service nova] Acquired lock "refresh_cache-3d4407f4-e078-4777-a110-cb3f922be2e8" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.network.neutron [req-71fdf42e-afaf-4c55-95fb-ef7a4d985a36 req-2df015a8-2e36-4b3a-98e4-d160ce0d8aa4 service nova] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Refreshing network info cache for port aec6d617-eb2b-42b3-8cce-f21de7c1b91c {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Start _get_guest_xml network_info=[{"id": "aec6d617-eb2b-42b3-8cce-f21de7c1b91c", "address": "fa:16:3e:92:a6:05", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapaec6d617-eb", "ovs_interfaceid": "aec6d617-eb2b-42b3-8cce-f21de7c1b91c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:38:46 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:38:46 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:38:42Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-752552685',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-752552685',id=24,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6e1fa352a8c144a1aada7ce0d8818429',ramdisk_id='',reservation_id='r-tlziajet',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-590302458',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:38:44Z,user_data=None,user_id='d99be569057f432484e4a60b46882bbd',uuid=3d4407f4-e078-4777-a110-cb3f922be2e8,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "aec6d617-eb2b-42b3-8cce-f21de7c1b91c", "address": "fa:16:3e:92:a6:05", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapaec6d617-eb", "ovs_interfaceid": "aec6d617-eb2b-42b3-8cce-f21de7c1b91c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converting VIF {"id": "aec6d617-eb2b-42b3-8cce-f21de7c1b91c", "address": "fa:16:3e:92:a6:05", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapaec6d617-eb", "ovs_interfaceid": "aec6d617-eb2b-42b3-8cce-f21de7c1b91c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:92:a6:05,bridge_name='br-int',has_traffic_filtering=True,id=aec6d617-eb2b-42b3-8cce-f21de7c1b91c,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapaec6d617-eb') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.objects.instance [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lazy-loading 'pci_devices' on Instance uuid 3d4407f4-e078-4777-a110-cb3f922be2e8 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:38:46 user nova-compute[72081]:   <uuid>3d4407f4-e078-4777-a110-cb3f922be2e8</uuid>
Apr 12 03:38:46 user nova-compute[72081]:   <name>instance-00000018</name>
Apr 12 03:38:46 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:38:46 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:38:46 user nova-compute[72081]:   <metadata>
Apr 12 03:38:46 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:38:46 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:38:46 user nova-compute[72081]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-752552685</nova:name>
Apr 12 03:38:46 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:38:46</nova:creationTime>
Apr 12 03:38:46 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:38:46 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:38:46 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:38:46 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:38:46 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:38:46 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:38:46 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:38:46 user nova-compute[72081]:       <nova:owner>
Apr 12 03:38:46 user nova-compute[72081]:         <nova:user uuid="d99be569057f432484e4a60b46882bbd">tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member</nova:user>
Apr 12 03:38:46 user nova-compute[72081]:         <nova:project uuid="6e1fa352a8c144a1aada7ce0d8818429">tempest-ServerBootFromVolumeStableRescueTest-590302458</nova:project>
Apr 12 03:38:46 user nova-compute[72081]:       </nova:owner>
Apr 12 03:38:46 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:38:46 user nova-compute[72081]:       <nova:ports>
Apr 12 03:38:46 user nova-compute[72081]:         <nova:port uuid="aec6d617-eb2b-42b3-8cce-f21de7c1b91c">
Apr 12 03:38:46 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.13" ipVersion="4"/>
Apr 12 03:38:46 user nova-compute[72081]:         </nova:port>
Apr 12 03:38:46 user nova-compute[72081]:       </nova:ports>
Apr 12 03:38:46 user nova-compute[72081]:     </nova:instance>
Apr 12 03:38:46 user nova-compute[72081]:   </metadata>
Apr 12 03:38:46 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:38:46 user nova-compute[72081]:     <system>
Apr 12 03:38:46 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:38:46 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:38:46 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:38:46 user nova-compute[72081]:       <entry name="serial">3d4407f4-e078-4777-a110-cb3f922be2e8</entry>
Apr 12 03:38:46 user nova-compute[72081]:       <entry name="uuid">3d4407f4-e078-4777-a110-cb3f922be2e8</entry>
Apr 12 03:38:46 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:38:46 user nova-compute[72081]:     </system>
Apr 12 03:38:46 user nova-compute[72081]:   </sysinfo>
Apr 12 03:38:46 user nova-compute[72081]:   <os>
Apr 12 03:38:46 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:38:46 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:38:46 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:38:46 user nova-compute[72081]:   </os>
Apr 12 03:38:46 user nova-compute[72081]:   <features>
Apr 12 03:38:46 user nova-compute[72081]:     <acpi/>
Apr 12 03:38:46 user nova-compute[72081]:     <apic/>
Apr 12 03:38:46 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:38:46 user nova-compute[72081]:   </features>
Apr 12 03:38:46 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:38:46 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:38:46 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:38:46 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:38:46 user nova-compute[72081]:   </clock>
Apr 12 03:38:46 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:38:46 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:38:46 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:38:46 user nova-compute[72081]:   </cpu>
Apr 12 03:38:46 user nova-compute[72081]:   <devices>
Apr 12 03:38:46 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:38:46 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:38:46 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk"/>
Apr 12 03:38:46 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:38:46 user nova-compute[72081]:     </disk>
Apr 12 03:38:46 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:38:46 user nova-compute[72081]:       <mac address="fa:16:3e:92:a6:05"/>
Apr 12 03:38:46 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:38:46 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:38:46 user nova-compute[72081]:       <target dev="tapaec6d617-eb"/>
Apr 12 03:38:46 user nova-compute[72081]:     </interface>
Apr 12 03:38:46 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:38:46 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/console.log" append="off"/>
Apr 12 03:38:46 user nova-compute[72081]:     </serial>
Apr 12 03:38:46 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:38:46 user nova-compute[72081]:     <video>
Apr 12 03:38:46 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:38:46 user nova-compute[72081]:     </video>
Apr 12 03:38:46 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:38:46 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:38:46 user nova-compute[72081]:     </rng>
Apr 12 03:38:46 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:38:46 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:38:46 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:38:46 user nova-compute[72081]:     </memballoon>
Apr 12 03:38:46 user nova-compute[72081]:   </devices>
Apr 12 03:38:46 user nova-compute[72081]: </domain>
Apr 12 03:38:46 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:38:42Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-752552685',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-752552685',id=24,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6e1fa352a8c144a1aada7ce0d8818429',ramdisk_id='',reservation_id='r-tlziajet',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-590302458',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:38:44Z,user_data=None,user_id='d99be569057f432484e4a60b46882bbd',uuid=3d4407f4-e078-4777-a110-cb3f922be2e8,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "aec6d617-eb2b-42b3-8cce-f21de7c1b91c", "address": "fa:16:3e:92:a6:05", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapaec6d617-eb", "ovs_interfaceid": "aec6d617-eb2b-42b3-8cce-f21de7c1b91c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converting VIF {"id": "aec6d617-eb2b-42b3-8cce-f21de7c1b91c", "address": "fa:16:3e:92:a6:05", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapaec6d617-eb", "ovs_interfaceid": "aec6d617-eb2b-42b3-8cce-f21de7c1b91c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:92:a6:05,bridge_name='br-int',has_traffic_filtering=True,id=aec6d617-eb2b-42b3-8cce-f21de7c1b91c,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapaec6d617-eb') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG os_vif [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:92:a6:05,bridge_name='br-int',has_traffic_filtering=True,id=aec6d617-eb2b-42b3-8cce-f21de7c1b91c,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapaec6d617-eb') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapaec6d617-eb, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapaec6d617-eb, col_values=(('external_ids', {'iface-id': 'aec6d617-eb2b-42b3-8cce-f21de7c1b91c', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:92:a6:05', 'vm-uuid': '3d4407f4-e078-4777-a110-cb3f922be2e8'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:46 user nova-compute[72081]: INFO os_vif [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:92:a6:05,bridge_name='br-int',has_traffic_filtering=True,id=aec6d617-eb2b-42b3-8cce-f21de7c1b91c,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapaec6d617-eb')
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] No VIF found with MAC fa:16:3e:92:a6:05, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.network.neutron [req-71fdf42e-afaf-4c55-95fb-ef7a4d985a36 req-2df015a8-2e36-4b3a-98e4-d160ce0d8aa4 service nova] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Updated VIF entry in instance network info cache for port aec6d617-eb2b-42b3-8cce-f21de7c1b91c. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG nova.network.neutron [req-71fdf42e-afaf-4c55-95fb-ef7a4d985a36 req-2df015a8-2e36-4b3a-98e4-d160ce0d8aa4 service nova] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Updating instance_info_cache with network_info: [{"id": "aec6d617-eb2b-42b3-8cce-f21de7c1b91c", "address": "fa:16:3e:92:a6:05", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapaec6d617-eb", "ovs_interfaceid": "aec6d617-eb2b-42b3-8cce-f21de7c1b91c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:38:46 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-71fdf42e-afaf-4c55-95fb-ef7a4d985a36 req-2df015a8-2e36-4b3a-98e4-d160ce0d8aa4 service nova] Releasing lock "refresh_cache-3d4407f4-e078-4777-a110-cb3f922be2e8" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:38:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:47 user nova-compute[72081]: DEBUG nova.compute.manager [req-a6f3e5e8-1818-425d-8f3f-53791e5891ab req-d7b77858-4c6e-44a4-a814-e585861adc9a service nova] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Received event network-vif-plugged-aec6d617-eb2b-42b3-8cce-f21de7c1b91c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:38:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a6f3e5e8-1818-425d-8f3f-53791e5891ab req-d7b77858-4c6e-44a4-a814-e585861adc9a service nova] Acquiring lock "3d4407f4-e078-4777-a110-cb3f922be2e8-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a6f3e5e8-1818-425d-8f3f-53791e5891ab req-d7b77858-4c6e-44a4-a814-e585861adc9a service nova] Lock "3d4407f4-e078-4777-a110-cb3f922be2e8-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a6f3e5e8-1818-425d-8f3f-53791e5891ab req-d7b77858-4c6e-44a4-a814-e585861adc9a service nova] Lock "3d4407f4-e078-4777-a110-cb3f922be2e8-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:47 user nova-compute[72081]: DEBUG nova.compute.manager [req-a6f3e5e8-1818-425d-8f3f-53791e5891ab req-d7b77858-4c6e-44a4-a814-e585861adc9a service nova] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] No waiting events found dispatching network-vif-plugged-aec6d617-eb2b-42b3-8cce-f21de7c1b91c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:38:47 user nova-compute[72081]: WARNING nova.compute.manager [req-a6f3e5e8-1818-425d-8f3f-53791e5891ab req-d7b77858-4c6e-44a4-a814-e585861adc9a service nova] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Received unexpected event network-vif-plugged-aec6d617-eb2b-42b3-8cce-f21de7c1b91c for instance with vm_state building and task_state spawning.
Apr 12 03:38:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ec0840cc-1c5a-4b41-bc91-0bb98f789892 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Acquiring lock "56834002-7593-4592-a86d-23bb90d00be0" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ec0840cc-1c5a-4b41-bc91-0bb98f789892 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "56834002-7593-4592-a86d-23bb90d00be0" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ec0840cc-1c5a-4b41-bc91-0bb98f789892 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Acquiring lock "56834002-7593-4592-a86d-23bb90d00be0-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ec0840cc-1c5a-4b41-bc91-0bb98f789892 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "56834002-7593-4592-a86d-23bb90d00be0-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ec0840cc-1c5a-4b41-bc91-0bb98f789892 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "56834002-7593-4592-a86d-23bb90d00be0-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:50 user nova-compute[72081]: INFO nova.compute.manager [None req-ec0840cc-1c5a-4b41-bc91-0bb98f789892 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Terminating instance
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ec0840cc-1c5a-4b41-bc91-0bb98f789892 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.compute.manager [req-29675f90-6324-47fd-8364-5bbffb55d952 req-1fe9c6af-1be3-40f0-9001-5e92963c233d service nova] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Received event network-vif-plugged-aec6d617-eb2b-42b3-8cce-f21de7c1b91c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-29675f90-6324-47fd-8364-5bbffb55d952 req-1fe9c6af-1be3-40f0-9001-5e92963c233d service nova] Acquiring lock "3d4407f4-e078-4777-a110-cb3f922be2e8-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-29675f90-6324-47fd-8364-5bbffb55d952 req-1fe9c6af-1be3-40f0-9001-5e92963c233d service nova] Lock "3d4407f4-e078-4777-a110-cb3f922be2e8-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-29675f90-6324-47fd-8364-5bbffb55d952 req-1fe9c6af-1be3-40f0-9001-5e92963c233d service nova] Lock "3d4407f4-e078-4777-a110-cb3f922be2e8-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.compute.manager [req-29675f90-6324-47fd-8364-5bbffb55d952 req-1fe9c6af-1be3-40f0-9001-5e92963c233d service nova] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] No waiting events found dispatching network-vif-plugged-aec6d617-eb2b-42b3-8cce-f21de7c1b91c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:38:50 user nova-compute[72081]: WARNING nova.compute.manager [req-29675f90-6324-47fd-8364-5bbffb55d952 req-1fe9c6af-1be3-40f0-9001-5e92963c233d service nova] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Received unexpected event network-vif-plugged-aec6d617-eb2b-42b3-8cce-f21de7c1b91c for instance with vm_state building and task_state spawning.
Apr 12 03:38:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.compute.manager [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270730.0812104, 3d4407f4-e078-4777-a110-cb3f922be2e8 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:38:50 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] VM Resumed (Lifecycle Event)
Apr 12 03:38:50 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Instance spawned successfully.
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:38:50 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270730.0813599, 3d4407f4-e078-4777-a110-cb3f922be2e8 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:38:50 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] VM Started (Lifecycle Event)
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:38:50 user nova-compute[72081]: INFO nova.compute.manager [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Took 6.17 seconds to spawn the instance on the hypervisor.
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.compute.manager [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:38:50 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.compute.manager [req-d46efd5c-f113-4e7a-8324-999c701e0267 req-fa2f9b32-1ee5-44ef-837c-3c13187a1d6b service nova] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Received event network-vif-unplugged-8f7b221a-25ae-41f9-847d-5aedf2d93a71 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d46efd5c-f113-4e7a-8324-999c701e0267 req-fa2f9b32-1ee5-44ef-837c-3c13187a1d6b service nova] Acquiring lock "56834002-7593-4592-a86d-23bb90d00be0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d46efd5c-f113-4e7a-8324-999c701e0267 req-fa2f9b32-1ee5-44ef-837c-3c13187a1d6b service nova] Lock "56834002-7593-4592-a86d-23bb90d00be0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d46efd5c-f113-4e7a-8324-999c701e0267 req-fa2f9b32-1ee5-44ef-837c-3c13187a1d6b service nova] Lock "56834002-7593-4592-a86d-23bb90d00be0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.compute.manager [req-d46efd5c-f113-4e7a-8324-999c701e0267 req-fa2f9b32-1ee5-44ef-837c-3c13187a1d6b service nova] [instance: 56834002-7593-4592-a86d-23bb90d00be0] No waiting events found dispatching network-vif-unplugged-8f7b221a-25ae-41f9-847d-5aedf2d93a71 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.compute.manager [req-d46efd5c-f113-4e7a-8324-999c701e0267 req-fa2f9b32-1ee5-44ef-837c-3c13187a1d6b service nova] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Received event network-vif-unplugged-8f7b221a-25ae-41f9-847d-5aedf2d93a71 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:38:50 user nova-compute[72081]: INFO nova.compute.manager [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Took 7.01 seconds to build instance.
Apr 12 03:38:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-5bc92262-fb64-42b7-9dd9-54482883ebd5 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "3d4407f4-e078-4777-a110-cb3f922be2e8" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.114s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:50 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Instance destroyed successfully.
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.objects.instance [None req-ec0840cc-1c5a-4b41-bc91-0bb98f789892 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lazy-loading 'resources' on Instance uuid 56834002-7593-4592-a86d-23bb90d00be0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-ec0840cc-1c5a-4b41-bc91-0bb98f789892 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:30:08Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-2007129702',display_name='tempest-ServersNegativeTestJSON-server-2007129702',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-2007129702',id=1,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:30:28Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='83a7ef3f97fd47849973d4e4482fb5f9',ramdisk_id='',reservation_id='r-qls8y0ir',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServersNegativeTestJSON-864113759',owner_user_name='tempest-ServersNegativeTestJSON-864113759-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:30:28Z,user_data=None,user_id='3da7394c51664a7caacfda6b7dda9066',uuid=56834002-7593-4592-a86d-23bb90d00be0,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "8f7b221a-25ae-41f9-847d-5aedf2d93a71", "address": "fa:16:3e:e7:22:1b", "network": {"id": "1a483654-b740-4544-84c9-53a41c11fa88", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1141588183-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "83a7ef3f97fd47849973d4e4482fb5f9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f7b221a-25", "ovs_interfaceid": "8f7b221a-25ae-41f9-847d-5aedf2d93a71", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-ec0840cc-1c5a-4b41-bc91-0bb98f789892 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Converting VIF {"id": "8f7b221a-25ae-41f9-847d-5aedf2d93a71", "address": "fa:16:3e:e7:22:1b", "network": {"id": "1a483654-b740-4544-84c9-53a41c11fa88", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1141588183-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "83a7ef3f97fd47849973d4e4482fb5f9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f7b221a-25", "ovs_interfaceid": "8f7b221a-25ae-41f9-847d-5aedf2d93a71", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-ec0840cc-1c5a-4b41-bc91-0bb98f789892 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:e7:22:1b,bridge_name='br-int',has_traffic_filtering=True,id=8f7b221a-25ae-41f9-847d-5aedf2d93a71,network=Network(1a483654-b740-4544-84c9-53a41c11fa88),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f7b221a-25') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG os_vif [None req-ec0840cc-1c5a-4b41-bc91-0bb98f789892 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:e7:22:1b,bridge_name='br-int',has_traffic_filtering=True,id=8f7b221a-25ae-41f9-847d-5aedf2d93a71,network=Network(1a483654-b740-4544-84c9-53a41c11fa88),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f7b221a-25') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap8f7b221a-25, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:50 user nova-compute[72081]: INFO os_vif [None req-ec0840cc-1c5a-4b41-bc91-0bb98f789892 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:e7:22:1b,bridge_name='br-int',has_traffic_filtering=True,id=8f7b221a-25ae-41f9-847d-5aedf2d93a71,network=Network(1a483654-b740-4544-84c9-53a41c11fa88),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f7b221a-25')
Apr 12 03:38:50 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-ec0840cc-1c5a-4b41-bc91-0bb98f789892 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Deleting instance files /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0_del
Apr 12 03:38:50 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-ec0840cc-1c5a-4b41-bc91-0bb98f789892 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Deletion of /opt/stack/data/nova/instances/56834002-7593-4592-a86d-23bb90d00be0_del complete
Apr 12 03:38:50 user nova-compute[72081]: INFO nova.compute.manager [None req-ec0840cc-1c5a-4b41-bc91-0bb98f789892 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Took 0.90 seconds to destroy the instance on the hypervisor.
Apr 12 03:38:50 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-ec0840cc-1c5a-4b41-bc91-0bb98f789892 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 56834002-7593-4592-a86d-23bb90d00be0] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Acquiring lock "bf03ba16-e423-4042-a8f8-3ca2d0055617" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "bf03ba16-e423-4042-a8f8-3ca2d0055617" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:50 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:38:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:51 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:38:51 user nova-compute[72081]: INFO nova.compute.claims [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Claim successful on node user
Apr 12 03:38:51 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:38:51 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Took 0.52 seconds to deallocate network for instance.
Apr 12 03:38:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ec0840cc-1c5a-4b41-bc91-0bb98f789892 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:51 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:38:51 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:38:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.470s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:51 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:38:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ec0840cc-1c5a-4b41-bc91-0bb98f789892 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.074s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:51 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:38:51 user nova-compute[72081]: DEBUG nova.network.neutron [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:38:51 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:38:51 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:38:51 user nova-compute[72081]: DEBUG nova.policy [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '45fe6fed1a5a43e5bd0925b92486ec68', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4f984af8c7048c7ba68c1530be9cc8d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:38:51 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:38:51 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:38:51 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Creating image(s)
Apr 12 03:38:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Acquiring lock "/opt/stack/data/nova/instances/bf03ba16-e423-4042-a8f8-3ca2d0055617/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "/opt/stack/data/nova/instances/bf03ba16-e423-4042-a8f8-3ca2d0055617/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "/opt/stack/data/nova/instances/bf03ba16-e423-4042-a8f8-3ca2d0055617/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.150s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:51 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-ec0840cc-1c5a-4b41-bc91-0bb98f789892 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:38:51 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-ec0840cc-1c5a-4b41-bc91-0bb98f789892 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:38:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.143s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/bf03ba16-e423-4042-a8f8-3ca2d0055617/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ec0840cc-1c5a-4b41-bc91-0bb98f789892 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.561s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/bf03ba16-e423-4042-a8f8-3ca2d0055617/disk 1073741824" returned: 0 in 0.078s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.223s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:52 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-ec0840cc-1c5a-4b41-bc91-0bb98f789892 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Deleted allocations for instance 56834002-7593-4592-a86d-23bb90d00be0
Apr 12 03:38:52 user nova-compute[72081]: DEBUG nova.compute.manager [req-5918d15b-1472-433c-9165-598ca4a27eae req-200210a6-495e-4a19-b714-4863714cd44b service nova] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Received event network-vif-deleted-8f7b221a-25ae-41f9-847d-5aedf2d93a71 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:38:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ec0840cc-1c5a-4b41-bc91-0bb98f789892 tempest-ServersNegativeTestJSON-864113759 tempest-ServersNegativeTestJSON-864113759-project-member] Lock "56834002-7593-4592-a86d-23bb90d00be0" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.293s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.154s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:52 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Checking if we can resize image /opt/stack/data/nova/instances/bf03ba16-e423-4042-a8f8-3ca2d0055617/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:38:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bf03ba16-e423-4042-a8f8-3ca2d0055617/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:38:52 user nova-compute[72081]: DEBUG nova.compute.manager [req-0a3f78ba-ef19-4961-985e-a8ba7f7e9c65 req-96270fd5-416c-4ef8-a330-8354d0f19fa8 service nova] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Received event network-vif-plugged-8f7b221a-25ae-41f9-847d-5aedf2d93a71 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:38:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0a3f78ba-ef19-4961-985e-a8ba7f7e9c65 req-96270fd5-416c-4ef8-a330-8354d0f19fa8 service nova] Acquiring lock "56834002-7593-4592-a86d-23bb90d00be0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0a3f78ba-ef19-4961-985e-a8ba7f7e9c65 req-96270fd5-416c-4ef8-a330-8354d0f19fa8 service nova] Lock "56834002-7593-4592-a86d-23bb90d00be0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0a3f78ba-ef19-4961-985e-a8ba7f7e9c65 req-96270fd5-416c-4ef8-a330-8354d0f19fa8 service nova] Lock "56834002-7593-4592-a86d-23bb90d00be0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:52 user nova-compute[72081]: DEBUG nova.compute.manager [req-0a3f78ba-ef19-4961-985e-a8ba7f7e9c65 req-96270fd5-416c-4ef8-a330-8354d0f19fa8 service nova] [instance: 56834002-7593-4592-a86d-23bb90d00be0] No waiting events found dispatching network-vif-plugged-8f7b221a-25ae-41f9-847d-5aedf2d93a71 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:38:52 user nova-compute[72081]: WARNING nova.compute.manager [req-0a3f78ba-ef19-4961-985e-a8ba7f7e9c65 req-96270fd5-416c-4ef8-a330-8354d0f19fa8 service nova] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Received unexpected event network-vif-plugged-8f7b221a-25ae-41f9-847d-5aedf2d93a71 for instance with vm_state deleted and task_state None.
Apr 12 03:38:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bf03ba16-e423-4042-a8f8-3ca2d0055617/disk --force-share --output=json" returned: 0 in 0.162s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:38:52 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Cannot resize image /opt/stack/data/nova/instances/bf03ba16-e423-4042-a8f8-3ca2d0055617/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:38:52 user nova-compute[72081]: DEBUG nova.objects.instance [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lazy-loading 'migration_context' on Instance uuid bf03ba16-e423-4042-a8f8-3ca2d0055617 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:38:52 user nova-compute[72081]: DEBUG nova.network.neutron [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Successfully created port: f258f181-9892-48fa-b054-aff7b58bd433 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:38:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:38:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Ensure instance console log exists: /opt/stack/data/nova/instances/bf03ba16-e423-4042-a8f8-3ca2d0055617/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:38:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.network.neutron [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Successfully updated port: f258f181-9892-48fa-b054-aff7b58bd433 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Acquiring lock "refresh_cache-bf03ba16-e423-4042-a8f8-3ca2d0055617" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Acquired lock "refresh_cache-bf03ba16-e423-4042-a8f8-3ca2d0055617" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.network.neutron [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.network.neutron [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.network.neutron [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Updating instance_info_cache with network_info: [{"id": "f258f181-9892-48fa-b054-aff7b58bd433", "address": "fa:16:3e:ab:86:b7", "network": {"id": "3ec59075-3409-4372-a0ca-6283a5d56ca0", "bridge": "br-int", "label": "tempest-VolumesActionsTest-959291386-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f4f984af8c7048c7ba68c1530be9cc8d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf258f181-98", "ovs_interfaceid": "f258f181-9892-48fa-b054-aff7b58bd433", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Releasing lock "refresh_cache-bf03ba16-e423-4042-a8f8-3ca2d0055617" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Instance network_info: |[{"id": "f258f181-9892-48fa-b054-aff7b58bd433", "address": "fa:16:3e:ab:86:b7", "network": {"id": "3ec59075-3409-4372-a0ca-6283a5d56ca0", "bridge": "br-int", "label": "tempest-VolumesActionsTest-959291386-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f4f984af8c7048c7ba68c1530be9cc8d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf258f181-98", "ovs_interfaceid": "f258f181-9892-48fa-b054-aff7b58bd433", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Start _get_guest_xml network_info=[{"id": "f258f181-9892-48fa-b054-aff7b58bd433", "address": "fa:16:3e:ab:86:b7", "network": {"id": "3ec59075-3409-4372-a0ca-6283a5d56ca0", "bridge": "br-int", "label": "tempest-VolumesActionsTest-959291386-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f4f984af8c7048c7ba68c1530be9cc8d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf258f181-98", "ovs_interfaceid": "f258f181-9892-48fa-b054-aff7b58bd433", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:38:53 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:38:53 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:38:50Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-1208948809',display_name='tempest-VolumesActionsTest-instance-1208948809',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-1208948809',id=25,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f4f984af8c7048c7ba68c1530be9cc8d',ramdisk_id='',reservation_id='r-f0z4bivk',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-629816748',owner_user_name='tempest-VolumesActionsTest-629816748-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:38:52Z,user_data=None,user_id='45fe6fed1a5a43e5bd0925b92486ec68',uuid=bf03ba16-e423-4042-a8f8-3ca2d0055617,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "f258f181-9892-48fa-b054-aff7b58bd433", "address": "fa:16:3e:ab:86:b7", "network": {"id": "3ec59075-3409-4372-a0ca-6283a5d56ca0", "bridge": "br-int", "label": "tempest-VolumesActionsTest-959291386-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f4f984af8c7048c7ba68c1530be9cc8d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf258f181-98", "ovs_interfaceid": "f258f181-9892-48fa-b054-aff7b58bd433", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Converting VIF {"id": "f258f181-9892-48fa-b054-aff7b58bd433", "address": "fa:16:3e:ab:86:b7", "network": {"id": "3ec59075-3409-4372-a0ca-6283a5d56ca0", "bridge": "br-int", "label": "tempest-VolumesActionsTest-959291386-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f4f984af8c7048c7ba68c1530be9cc8d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf258f181-98", "ovs_interfaceid": "f258f181-9892-48fa-b054-aff7b58bd433", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ab:86:b7,bridge_name='br-int',has_traffic_filtering=True,id=f258f181-9892-48fa-b054-aff7b58bd433,network=Network(3ec59075-3409-4372-a0ca-6283a5d56ca0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf258f181-98') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.objects.instance [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lazy-loading 'pci_devices' on Instance uuid bf03ba16-e423-4042-a8f8-3ca2d0055617 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:38:53 user nova-compute[72081]:   <uuid>bf03ba16-e423-4042-a8f8-3ca2d0055617</uuid>
Apr 12 03:38:53 user nova-compute[72081]:   <name>instance-00000019</name>
Apr 12 03:38:53 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:38:53 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:38:53 user nova-compute[72081]:   <metadata>
Apr 12 03:38:53 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:38:53 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:38:53 user nova-compute[72081]:       <nova:name>tempest-VolumesActionsTest-instance-1208948809</nova:name>
Apr 12 03:38:53 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:38:53</nova:creationTime>
Apr 12 03:38:53 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:38:53 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:38:53 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:38:53 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:38:53 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:38:53 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:38:53 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:38:53 user nova-compute[72081]:       <nova:owner>
Apr 12 03:38:53 user nova-compute[72081]:         <nova:user uuid="45fe6fed1a5a43e5bd0925b92486ec68">tempest-VolumesActionsTest-629816748-project-member</nova:user>
Apr 12 03:38:53 user nova-compute[72081]:         <nova:project uuid="f4f984af8c7048c7ba68c1530be9cc8d">tempest-VolumesActionsTest-629816748</nova:project>
Apr 12 03:38:53 user nova-compute[72081]:       </nova:owner>
Apr 12 03:38:53 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:38:53 user nova-compute[72081]:       <nova:ports>
Apr 12 03:38:53 user nova-compute[72081]:         <nova:port uuid="f258f181-9892-48fa-b054-aff7b58bd433">
Apr 12 03:38:53 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.13" ipVersion="4"/>
Apr 12 03:38:53 user nova-compute[72081]:         </nova:port>
Apr 12 03:38:53 user nova-compute[72081]:       </nova:ports>
Apr 12 03:38:53 user nova-compute[72081]:     </nova:instance>
Apr 12 03:38:53 user nova-compute[72081]:   </metadata>
Apr 12 03:38:53 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:38:53 user nova-compute[72081]:     <system>
Apr 12 03:38:53 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:38:53 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:38:53 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:38:53 user nova-compute[72081]:       <entry name="serial">bf03ba16-e423-4042-a8f8-3ca2d0055617</entry>
Apr 12 03:38:53 user nova-compute[72081]:       <entry name="uuid">bf03ba16-e423-4042-a8f8-3ca2d0055617</entry>
Apr 12 03:38:53 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:38:53 user nova-compute[72081]:     </system>
Apr 12 03:38:53 user nova-compute[72081]:   </sysinfo>
Apr 12 03:38:53 user nova-compute[72081]:   <os>
Apr 12 03:38:53 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:38:53 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:38:53 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:38:53 user nova-compute[72081]:   </os>
Apr 12 03:38:53 user nova-compute[72081]:   <features>
Apr 12 03:38:53 user nova-compute[72081]:     <acpi/>
Apr 12 03:38:53 user nova-compute[72081]:     <apic/>
Apr 12 03:38:53 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:38:53 user nova-compute[72081]:   </features>
Apr 12 03:38:53 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:38:53 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:38:53 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:38:53 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:38:53 user nova-compute[72081]:   </clock>
Apr 12 03:38:53 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:38:53 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:38:53 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:38:53 user nova-compute[72081]:   </cpu>
Apr 12 03:38:53 user nova-compute[72081]:   <devices>
Apr 12 03:38:53 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:38:53 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:38:53 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/bf03ba16-e423-4042-a8f8-3ca2d0055617/disk"/>
Apr 12 03:38:53 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:38:53 user nova-compute[72081]:     </disk>
Apr 12 03:38:53 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:38:53 user nova-compute[72081]:       <mac address="fa:16:3e:ab:86:b7"/>
Apr 12 03:38:53 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:38:53 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:38:53 user nova-compute[72081]:       <target dev="tapf258f181-98"/>
Apr 12 03:38:53 user nova-compute[72081]:     </interface>
Apr 12 03:38:53 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:38:53 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/bf03ba16-e423-4042-a8f8-3ca2d0055617/console.log" append="off"/>
Apr 12 03:38:53 user nova-compute[72081]:     </serial>
Apr 12 03:38:53 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:38:53 user nova-compute[72081]:     <video>
Apr 12 03:38:53 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:38:53 user nova-compute[72081]:     </video>
Apr 12 03:38:53 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:38:53 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:38:53 user nova-compute[72081]:     </rng>
Apr 12 03:38:53 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:38:53 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:38:53 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:38:53 user nova-compute[72081]:     </memballoon>
Apr 12 03:38:53 user nova-compute[72081]:   </devices>
Apr 12 03:38:53 user nova-compute[72081]: </domain>
Apr 12 03:38:53 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:38:50Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-1208948809',display_name='tempest-VolumesActionsTest-instance-1208948809',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-1208948809',id=25,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f4f984af8c7048c7ba68c1530be9cc8d',ramdisk_id='',reservation_id='r-f0z4bivk',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-629816748',owner_user_name='tempest-VolumesActionsTest-629816748-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:38:52Z,user_data=None,user_id='45fe6fed1a5a43e5bd0925b92486ec68',uuid=bf03ba16-e423-4042-a8f8-3ca2d0055617,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "f258f181-9892-48fa-b054-aff7b58bd433", "address": "fa:16:3e:ab:86:b7", "network": {"id": "3ec59075-3409-4372-a0ca-6283a5d56ca0", "bridge": "br-int", "label": "tempest-VolumesActionsTest-959291386-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f4f984af8c7048c7ba68c1530be9cc8d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf258f181-98", "ovs_interfaceid": "f258f181-9892-48fa-b054-aff7b58bd433", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Converting VIF {"id": "f258f181-9892-48fa-b054-aff7b58bd433", "address": "fa:16:3e:ab:86:b7", "network": {"id": "3ec59075-3409-4372-a0ca-6283a5d56ca0", "bridge": "br-int", "label": "tempest-VolumesActionsTest-959291386-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f4f984af8c7048c7ba68c1530be9cc8d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf258f181-98", "ovs_interfaceid": "f258f181-9892-48fa-b054-aff7b58bd433", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ab:86:b7,bridge_name='br-int',has_traffic_filtering=True,id=f258f181-9892-48fa-b054-aff7b58bd433,network=Network(3ec59075-3409-4372-a0ca-6283a5d56ca0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf258f181-98') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG os_vif [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:ab:86:b7,bridge_name='br-int',has_traffic_filtering=True,id=f258f181-9892-48fa-b054-aff7b58bd433,network=Network(3ec59075-3409-4372-a0ca-6283a5d56ca0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf258f181-98') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapf258f181-98, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapf258f181-98, col_values=(('external_ids', {'iface-id': 'f258f181-9892-48fa-b054-aff7b58bd433', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:ab:86:b7', 'vm-uuid': 'bf03ba16-e423-4042-a8f8-3ca2d0055617'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:53 user nova-compute[72081]: INFO os_vif [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:ab:86:b7,bridge_name='br-int',has_traffic_filtering=True,id=f258f181-9892-48fa-b054-aff7b58bd433,network=Network(3ec59075-3409-4372-a0ca-6283a5d56ca0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf258f181-98')
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:38:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] No VIF found with MAC fa:16:3e:ab:86:b7, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:38:54 user nova-compute[72081]: DEBUG nova.compute.manager [req-9a5b1913-c860-411a-a1f3-5980439beb30 req-1a2e456e-cbad-4d05-9657-ac89a358eb79 service nova] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Received event network-changed-f258f181-9892-48fa-b054-aff7b58bd433 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:38:54 user nova-compute[72081]: DEBUG nova.compute.manager [req-9a5b1913-c860-411a-a1f3-5980439beb30 req-1a2e456e-cbad-4d05-9657-ac89a358eb79 service nova] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Refreshing instance network info cache due to event network-changed-f258f181-9892-48fa-b054-aff7b58bd433. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:38:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9a5b1913-c860-411a-a1f3-5980439beb30 req-1a2e456e-cbad-4d05-9657-ac89a358eb79 service nova] Acquiring lock "refresh_cache-bf03ba16-e423-4042-a8f8-3ca2d0055617" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:38:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9a5b1913-c860-411a-a1f3-5980439beb30 req-1a2e456e-cbad-4d05-9657-ac89a358eb79 service nova] Acquired lock "refresh_cache-bf03ba16-e423-4042-a8f8-3ca2d0055617" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:38:54 user nova-compute[72081]: DEBUG nova.network.neutron [req-9a5b1913-c860-411a-a1f3-5980439beb30 req-1a2e456e-cbad-4d05-9657-ac89a358eb79 service nova] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Refreshing network info cache for port f258f181-9892-48fa-b054-aff7b58bd433 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:38:54 user nova-compute[72081]: DEBUG nova.network.neutron [req-9a5b1913-c860-411a-a1f3-5980439beb30 req-1a2e456e-cbad-4d05-9657-ac89a358eb79 service nova] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Updated VIF entry in instance network info cache for port f258f181-9892-48fa-b054-aff7b58bd433. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:38:54 user nova-compute[72081]: DEBUG nova.network.neutron [req-9a5b1913-c860-411a-a1f3-5980439beb30 req-1a2e456e-cbad-4d05-9657-ac89a358eb79 service nova] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Updating instance_info_cache with network_info: [{"id": "f258f181-9892-48fa-b054-aff7b58bd433", "address": "fa:16:3e:ab:86:b7", "network": {"id": "3ec59075-3409-4372-a0ca-6283a5d56ca0", "bridge": "br-int", "label": "tempest-VolumesActionsTest-959291386-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f4f984af8c7048c7ba68c1530be9cc8d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf258f181-98", "ovs_interfaceid": "f258f181-9892-48fa-b054-aff7b58bd433", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:38:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9a5b1913-c860-411a-a1f3-5980439beb30 req-1a2e456e-cbad-4d05-9657-ac89a358eb79 service nova] Releasing lock "refresh_cache-bf03ba16-e423-4042-a8f8-3ca2d0055617" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:38:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:55 user nova-compute[72081]: DEBUG nova.compute.manager [req-feeae2fb-c4da-48e5-be3e-d753a4d99435 req-ba75024d-22b8-4e12-b0df-c86bba30b2a1 service nova] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Received event network-vif-plugged-f258f181-9892-48fa-b054-aff7b58bd433 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:38:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-feeae2fb-c4da-48e5-be3e-d753a4d99435 req-ba75024d-22b8-4e12-b0df-c86bba30b2a1 service nova] Acquiring lock "bf03ba16-e423-4042-a8f8-3ca2d0055617-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-feeae2fb-c4da-48e5-be3e-d753a4d99435 req-ba75024d-22b8-4e12-b0df-c86bba30b2a1 service nova] Lock "bf03ba16-e423-4042-a8f8-3ca2d0055617-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-feeae2fb-c4da-48e5-be3e-d753a4d99435 req-ba75024d-22b8-4e12-b0df-c86bba30b2a1 service nova] Lock "bf03ba16-e423-4042-a8f8-3ca2d0055617-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:55 user nova-compute[72081]: DEBUG nova.compute.manager [req-feeae2fb-c4da-48e5-be3e-d753a4d99435 req-ba75024d-22b8-4e12-b0df-c86bba30b2a1 service nova] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] No waiting events found dispatching network-vif-plugged-f258f181-9892-48fa-b054-aff7b58bd433 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:38:55 user nova-compute[72081]: WARNING nova.compute.manager [req-feeae2fb-c4da-48e5-be3e-d753a4d99435 req-ba75024d-22b8-4e12-b0df-c86bba30b2a1 service nova] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Received unexpected event network-vif-plugged-f258f181-9892-48fa-b054-aff7b58bd433 for instance with vm_state building and task_state spawning.
Apr 12 03:38:57 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270737.1519618, bf03ba16-e423-4042-a8f8-3ca2d0055617 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:38:57 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] VM Resumed (Lifecycle Event)
Apr 12 03:38:57 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:38:57 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:38:57 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Instance spawned successfully.
Apr 12 03:38:57 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:38:57 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:38:57 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:38:57 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:57 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:57 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:57 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:57 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:57 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:38:57 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:38:57 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270737.1554353, bf03ba16-e423-4042-a8f8-3ca2d0055617 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:38:57 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] VM Started (Lifecycle Event)
Apr 12 03:38:57 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:38:57 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:38:57 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:38:57 user nova-compute[72081]: INFO nova.compute.manager [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Took 5.53 seconds to spawn the instance on the hypervisor.
Apr 12 03:38:57 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:38:57 user nova-compute[72081]: INFO nova.compute.manager [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Took 6.34 seconds to build instance.
Apr 12 03:38:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1a7d8570-597f-47cc-a481-4dac33f03845 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "bf03ba16-e423-4042-a8f8-3ca2d0055617" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.429s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:57 user nova-compute[72081]: DEBUG nova.compute.manager [req-60a7ce74-9238-44eb-82b5-4f2ca9a11e82 req-1dd4ef06-b67c-49cf-b6b8-16993d9a48db service nova] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Received event network-vif-plugged-f258f181-9892-48fa-b054-aff7b58bd433 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:38:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-60a7ce74-9238-44eb-82b5-4f2ca9a11e82 req-1dd4ef06-b67c-49cf-b6b8-16993d9a48db service nova] Acquiring lock "bf03ba16-e423-4042-a8f8-3ca2d0055617-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:38:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-60a7ce74-9238-44eb-82b5-4f2ca9a11e82 req-1dd4ef06-b67c-49cf-b6b8-16993d9a48db service nova] Lock "bf03ba16-e423-4042-a8f8-3ca2d0055617-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:38:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-60a7ce74-9238-44eb-82b5-4f2ca9a11e82 req-1dd4ef06-b67c-49cf-b6b8-16993d9a48db service nova] Lock "bf03ba16-e423-4042-a8f8-3ca2d0055617-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:38:57 user nova-compute[72081]: DEBUG nova.compute.manager [req-60a7ce74-9238-44eb-82b5-4f2ca9a11e82 req-1dd4ef06-b67c-49cf-b6b8-16993d9a48db service nova] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] No waiting events found dispatching network-vif-plugged-f258f181-9892-48fa-b054-aff7b58bd433 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:38:57 user nova-compute[72081]: WARNING nova.compute.manager [req-60a7ce74-9238-44eb-82b5-4f2ca9a11e82 req-1dd4ef06-b67c-49cf-b6b8-16993d9a48db service nova] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Received unexpected event network-vif-plugged-f258f181-9892-48fa-b054-aff7b58bd433 for instance with vm_state active and task_state None.
Apr 12 03:38:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:38:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:02 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:39:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:03 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:39:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:04 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:39:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:39:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:39:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:39:04 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:39:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdd (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:39:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bf03ba16-e423-4042-a8f8-3ca2d0055617/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bf03ba16-e423-4042-a8f8-3ca2d0055617/disk --force-share --output=json" returned: 0 in 0.166s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bf03ba16-e423-4042-a8f8-3ca2d0055617/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bf03ba16-e423-4042-a8f8-3ca2d0055617/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:05 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681270730.7536016, 56834002-7593-4592-a86d-23bb90d00be0 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:39:05 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 56834002-7593-4592-a86d-23bb90d00be0] VM Stopped (Lifecycle Event)
Apr 12 03:39:05 user nova-compute[72081]: DEBUG nova.compute.manager [None req-afc01fbf-c567-4a52-866c-faf6d4107a85 None None] [instance: 56834002-7593-4592-a86d-23bb90d00be0] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:39:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.127s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ff5a7005-6dd4-491a-a55c-db7a82dc28f7/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ff5a7005-6dd4-491a-a55c-db7a82dc28f7/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ff5a7005-6dd4-491a-a55c-db7a82dc28f7/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ff5a7005-6dd4-491a-a55c-db7a82dc28f7/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:39:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk --force-share --output=json" returned: 0 in 0.129s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:39:07 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:39:07 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:39:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=7595MB free_disk=26.085086822509766GB free_vcpus=1 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:39:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:39:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:39:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 2d0904c3-74e0-47bb-a832-758a7035b2f5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:39:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance bcd267d2-5b2c-4ed2-b634-0210fae94dfd actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:39:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 48977eb2-ce43-4577-a1cb-340e35703dbd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:39:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 23367b24-0a3b-454d-901d-8a3e5396333a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:39:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:39:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 06481254-722a-4053-a6af-11f5dcc68636 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:39:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 1eef2c91-dfad-43aa-b193-25970db83b79 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:39:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance ff5a7005-6dd4-491a-a55c-db7a82dc28f7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:39:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance f73fa883-75f0-4593-a89c-d009c6d4ebf0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:39:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 3d4407f4-e078-4777-a110-cb3f922be2e8 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:39:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance bf03ba16-e423-4042-a8f8-3ca2d0055617 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:39:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 11 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:39:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=1920MB phys_disk=40GB used_disk=9GB total_vcpus=12 used_vcpus=11 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:39:08 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:39:08 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:39:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:39:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.531s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:39:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:09 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:39:09 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:39:09 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:39:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:39:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:39:09 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 03:39:10 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Updating instance_info_cache with network_info: [{"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:39:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:39:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 03:39:10 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:39:10 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:39:10 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:39:10 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:39:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:39:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:22 user nova-compute[72081]: DEBUG nova.compute.manager [req-b14db339-b9c0-442f-97a0-22da422821e9 req-3ca21946-32d9-4325-93b5-953e3fdbc22f service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Received event network-changed-5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:39:22 user nova-compute[72081]: DEBUG nova.compute.manager [req-b14db339-b9c0-442f-97a0-22da422821e9 req-3ca21946-32d9-4325-93b5-953e3fdbc22f service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Refreshing instance network info cache due to event network-changed-5861aafc-4312-4d82-965c-2d1d993d8ea0. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:39:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b14db339-b9c0-442f-97a0-22da422821e9 req-3ca21946-32d9-4325-93b5-953e3fdbc22f service nova] Acquiring lock "refresh_cache-06481254-722a-4053-a6af-11f5dcc68636" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:39:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b14db339-b9c0-442f-97a0-22da422821e9 req-3ca21946-32d9-4325-93b5-953e3fdbc22f service nova] Acquired lock "refresh_cache-06481254-722a-4053-a6af-11f5dcc68636" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:39:22 user nova-compute[72081]: DEBUG nova.network.neutron [req-b14db339-b9c0-442f-97a0-22da422821e9 req-3ca21946-32d9-4325-93b5-953e3fdbc22f service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Refreshing network info cache for port 5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:39:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:23 user nova-compute[72081]: DEBUG nova.network.neutron [req-b14db339-b9c0-442f-97a0-22da422821e9 req-3ca21946-32d9-4325-93b5-953e3fdbc22f service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Updated VIF entry in instance network info cache for port 5861aafc-4312-4d82-965c-2d1d993d8ea0. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:39:23 user nova-compute[72081]: DEBUG nova.network.neutron [req-b14db339-b9c0-442f-97a0-22da422821e9 req-3ca21946-32d9-4325-93b5-953e3fdbc22f service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Updating instance_info_cache with network_info: [{"id": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "address": "fa:16:3e:c7:7c:eb", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.227", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5861aafc-43", "ovs_interfaceid": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:39:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b14db339-b9c0-442f-97a0-22da422821e9 req-3ca21946-32d9-4325-93b5-953e3fdbc22f service nova] Releasing lock "refresh_cache-06481254-722a-4053-a6af-11f5dcc68636" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:39:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0f6bb910-6eab-4356-ac52-fd62b47532e4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "23367b24-0a3b-454d-901d-8a3e5396333a" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:39:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0f6bb910-6eab-4356-ac52-fd62b47532e4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "23367b24-0a3b-454d-901d-8a3e5396333a" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:39:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0f6bb910-6eab-4356-ac52-fd62b47532e4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:39:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0f6bb910-6eab-4356-ac52-fd62b47532e4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:39:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0f6bb910-6eab-4356-ac52-fd62b47532e4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:39:23 user nova-compute[72081]: INFO nova.compute.manager [None req-0f6bb910-6eab-4356-ac52-fd62b47532e4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Terminating instance
Apr 12 03:39:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0f6bb910-6eab-4356-ac52-fd62b47532e4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:39:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:24 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Instance destroyed successfully.
Apr 12 03:39:24 user nova-compute[72081]: DEBUG nova.objects.instance [None req-0f6bb910-6eab-4356-ac52-fd62b47532e4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lazy-loading 'resources' on Instance uuid 23367b24-0a3b-454d-901d-8a3e5396333a {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:39:24 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-0f6bb910-6eab-4356-ac52-fd62b47532e4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:34:29Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-2135071440',display_name='tempest-TestMinimumBasicScenario-server-2135071440',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-2135071440',id=17,image_ref='5267358d-2283-43f2-a893-8842d5de70da',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMkexWkk4Rxu6k02iex2AsbJ+PDY5htmdDN8vQSba2S3zwrjAPkO8wlI5tbs0OLJWNWhrMe6/AoKb3eoA7zRDJCJlh8REZk8FLy3CbfuwbBsBPrnYnRJ2rfKKtaAGF256w==',key_name='tempest-TestMinimumBasicScenario-2058975683',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:34:38Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='05e1e25933cd42df98fb060b8d2701dc',ramdisk_id='',reservation_id='r-0n9gnxqj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='5267358d-2283-43f2-a893-8842d5de70da',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-TestMinimumBasicScenario-1605347971',owner_user_name='tempest-TestMinimumBasicScenario-1605347971-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:36:45Z,user_data=None,user_id='3f8024260bf14d9995e67cb0d1bc8ddc',uuid=23367b24-0a3b-454d-901d-8a3e5396333a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "a76c6f07-234b-4832-8ace-ec81656b06a7", "address": "fa:16:3e:e4:18:3e", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa76c6f07-23", "ovs_interfaceid": "a76c6f07-234b-4832-8ace-ec81656b06a7", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:39:24 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0f6bb910-6eab-4356-ac52-fd62b47532e4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Converting VIF {"id": "a76c6f07-234b-4832-8ace-ec81656b06a7", "address": "fa:16:3e:e4:18:3e", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa76c6f07-23", "ovs_interfaceid": "a76c6f07-234b-4832-8ace-ec81656b06a7", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:39:24 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-0f6bb910-6eab-4356-ac52-fd62b47532e4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:e4:18:3e,bridge_name='br-int',has_traffic_filtering=True,id=a76c6f07-234b-4832-8ace-ec81656b06a7,network=Network(91a7501a-44a5-4d48-a1f7-e1195ac99607),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa76c6f07-23') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:39:24 user nova-compute[72081]: DEBUG os_vif [None req-0f6bb910-6eab-4356-ac52-fd62b47532e4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:e4:18:3e,bridge_name='br-int',has_traffic_filtering=True,id=a76c6f07-234b-4832-8ace-ec81656b06a7,network=Network(91a7501a-44a5-4d48-a1f7-e1195ac99607),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa76c6f07-23') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:39:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapa76c6f07-23, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:39:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:39:24 user nova-compute[72081]: INFO os_vif [None req-0f6bb910-6eab-4356-ac52-fd62b47532e4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:e4:18:3e,bridge_name='br-int',has_traffic_filtering=True,id=a76c6f07-234b-4832-8ace-ec81656b06a7,network=Network(91a7501a-44a5-4d48-a1f7-e1195ac99607),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa76c6f07-23')
Apr 12 03:39:24 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-0f6bb910-6eab-4356-ac52-fd62b47532e4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Deleting instance files /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a_del
Apr 12 03:39:24 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-0f6bb910-6eab-4356-ac52-fd62b47532e4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Deletion of /opt/stack/data/nova/instances/23367b24-0a3b-454d-901d-8a3e5396333a_del complete
Apr 12 03:39:24 user nova-compute[72081]: INFO nova.compute.manager [None req-0f6bb910-6eab-4356-ac52-fd62b47532e4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Took 0.96 seconds to destroy the instance on the hypervisor.
Apr 12 03:39:24 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-0f6bb910-6eab-4356-ac52-fd62b47532e4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:39:24 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:39:24 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:39:24 user nova-compute[72081]: DEBUG nova.compute.manager [req-fee3ccc9-2690-4db2-9991-ff8ed3383720 req-43d65d6d-9406-4619-b386-1448003b80f8 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Received event network-vif-unplugged-a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:39:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-fee3ccc9-2690-4db2-9991-ff8ed3383720 req-43d65d6d-9406-4619-b386-1448003b80f8 service nova] Acquiring lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:39:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-fee3ccc9-2690-4db2-9991-ff8ed3383720 req-43d65d6d-9406-4619-b386-1448003b80f8 service nova] Lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:39:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-fee3ccc9-2690-4db2-9991-ff8ed3383720 req-43d65d6d-9406-4619-b386-1448003b80f8 service nova] Lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:39:24 user nova-compute[72081]: DEBUG nova.compute.manager [req-fee3ccc9-2690-4db2-9991-ff8ed3383720 req-43d65d6d-9406-4619-b386-1448003b80f8 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] No waiting events found dispatching network-vif-unplugged-a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:39:24 user nova-compute[72081]: DEBUG nova.compute.manager [req-fee3ccc9-2690-4db2-9991-ff8ed3383720 req-43d65d6d-9406-4619-b386-1448003b80f8 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Received event network-vif-unplugged-a76c6f07-234b-4832-8ace-ec81656b06a7 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:39:24 user nova-compute[72081]: DEBUG nova.compute.manager [req-fee3ccc9-2690-4db2-9991-ff8ed3383720 req-43d65d6d-9406-4619-b386-1448003b80f8 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Received event network-vif-plugged-a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:39:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-fee3ccc9-2690-4db2-9991-ff8ed3383720 req-43d65d6d-9406-4619-b386-1448003b80f8 service nova] Acquiring lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:39:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-fee3ccc9-2690-4db2-9991-ff8ed3383720 req-43d65d6d-9406-4619-b386-1448003b80f8 service nova] Lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:39:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-fee3ccc9-2690-4db2-9991-ff8ed3383720 req-43d65d6d-9406-4619-b386-1448003b80f8 service nova] Lock "23367b24-0a3b-454d-901d-8a3e5396333a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:39:24 user nova-compute[72081]: DEBUG nova.compute.manager [req-fee3ccc9-2690-4db2-9991-ff8ed3383720 req-43d65d6d-9406-4619-b386-1448003b80f8 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] No waiting events found dispatching network-vif-plugged-a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:39:24 user nova-compute[72081]: WARNING nova.compute.manager [req-fee3ccc9-2690-4db2-9991-ff8ed3383720 req-43d65d6d-9406-4619-b386-1448003b80f8 service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Received unexpected event network-vif-plugged-a76c6f07-234b-4832-8ace-ec81656b06a7 for instance with vm_state active and task_state deleting.
Apr 12 03:39:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "06481254-722a-4053-a6af-11f5dcc68636" by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:39:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "06481254-722a-4053-a6af-11f5dcc68636" acquired by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:39:25 user nova-compute[72081]: INFO nova.compute.manager [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Shelving
Apr 12 03:39:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:25 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:39:25 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Took 0.74 seconds to deallocate network for instance.
Apr 12 03:39:25 user nova-compute[72081]: DEBUG nova.compute.manager [req-156bf05a-06e5-466b-aad7-139729f31abf req-5d4d16b8-5778-4ff1-970a-d22dd0b249de service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Received event network-vif-deleted-a76c6f07-234b-4832-8ace-ec81656b06a7 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:39:25 user nova-compute[72081]: INFO nova.compute.manager [req-156bf05a-06e5-466b-aad7-139729f31abf req-5d4d16b8-5778-4ff1-970a-d22dd0b249de service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Neutron deleted interface a76c6f07-234b-4832-8ace-ec81656b06a7; detaching it from the instance and deleting it from the info cache
Apr 12 03:39:25 user nova-compute[72081]: DEBUG nova.network.neutron [req-156bf05a-06e5-466b-aad7-139729f31abf req-5d4d16b8-5778-4ff1-970a-d22dd0b249de service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:39:25 user nova-compute[72081]: DEBUG nova.compute.manager [req-156bf05a-06e5-466b-aad7-139729f31abf req-5d4d16b8-5778-4ff1-970a-d22dd0b249de service nova] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Detach interface failed, port_id=a76c6f07-234b-4832-8ace-ec81656b06a7, reason: Instance 23367b24-0a3b-454d-901d-8a3e5396333a could not be found. {{(pid=72081) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 12 03:39:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0f6bb910-6eab-4356-ac52-fd62b47532e4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:39:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0f6bb910-6eab-4356-ac52-fd62b47532e4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:39:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:25 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-0f6bb910-6eab-4356-ac52-fd62b47532e4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:39:25 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-0f6bb910-6eab-4356-ac52-fd62b47532e4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:39:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0f6bb910-6eab-4356-ac52-fd62b47532e4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.410s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:39:25 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-0f6bb910-6eab-4356-ac52-fd62b47532e4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Deleted allocations for instance 23367b24-0a3b-454d-901d-8a3e5396333a
Apr 12 03:39:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-0f6bb910-6eab-4356-ac52-fd62b47532e4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "23367b24-0a3b-454d-901d-8a3e5396333a" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.345s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:39:26 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Instance destroyed successfully.
Apr 12 03:39:26 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Beginning cold snapshot process
Apr 12 03:39:26 user nova-compute[72081]: DEBUG nova.privsep.utils [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72081) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 12 03:39:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk /opt/stack/data/nova/instances/snapshots/tmp56hz5xii/53fad05e096443bf8a01f1e3941a8f81 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk /opt/stack/data/nova/instances/snapshots/tmp56hz5xii/53fad05e096443bf8a01f1e3941a8f81" returned: 0 in 0.762s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:27 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Snapshot extracted, beginning image upload
Apr 12 03:39:27 user nova-compute[72081]: DEBUG nova.compute.manager [req-b0827922-7004-48b0-8e85-90061c31d719 req-e85644eb-4b31-4b1b-b459-aaf602b54e2f service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Received event network-vif-unplugged-5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:39:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b0827922-7004-48b0-8e85-90061c31d719 req-e85644eb-4b31-4b1b-b459-aaf602b54e2f service nova] Acquiring lock "06481254-722a-4053-a6af-11f5dcc68636-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:39:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b0827922-7004-48b0-8e85-90061c31d719 req-e85644eb-4b31-4b1b-b459-aaf602b54e2f service nova] Lock "06481254-722a-4053-a6af-11f5dcc68636-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:39:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b0827922-7004-48b0-8e85-90061c31d719 req-e85644eb-4b31-4b1b-b459-aaf602b54e2f service nova] Lock "06481254-722a-4053-a6af-11f5dcc68636-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:39:27 user nova-compute[72081]: DEBUG nova.compute.manager [req-b0827922-7004-48b0-8e85-90061c31d719 req-e85644eb-4b31-4b1b-b459-aaf602b54e2f service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] No waiting events found dispatching network-vif-unplugged-5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:39:27 user nova-compute[72081]: WARNING nova.compute.manager [req-b0827922-7004-48b0-8e85-90061c31d719 req-e85644eb-4b31-4b1b-b459-aaf602b54e2f service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Received unexpected event network-vif-unplugged-5861aafc-4312-4d82-965c-2d1d993d8ea0 for instance with vm_state active and task_state shelving_image_uploading.
Apr 12 03:39:27 user nova-compute[72081]: DEBUG nova.compute.manager [req-b0827922-7004-48b0-8e85-90061c31d719 req-e85644eb-4b31-4b1b-b459-aaf602b54e2f service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Received event network-vif-plugged-5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:39:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b0827922-7004-48b0-8e85-90061c31d719 req-e85644eb-4b31-4b1b-b459-aaf602b54e2f service nova] Acquiring lock "06481254-722a-4053-a6af-11f5dcc68636-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:39:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b0827922-7004-48b0-8e85-90061c31d719 req-e85644eb-4b31-4b1b-b459-aaf602b54e2f service nova] Lock "06481254-722a-4053-a6af-11f5dcc68636-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:39:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b0827922-7004-48b0-8e85-90061c31d719 req-e85644eb-4b31-4b1b-b459-aaf602b54e2f service nova] Lock "06481254-722a-4053-a6af-11f5dcc68636-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:39:27 user nova-compute[72081]: DEBUG nova.compute.manager [req-b0827922-7004-48b0-8e85-90061c31d719 req-e85644eb-4b31-4b1b-b459-aaf602b54e2f service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] No waiting events found dispatching network-vif-plugged-5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:39:27 user nova-compute[72081]: WARNING nova.compute.manager [req-b0827922-7004-48b0-8e85-90061c31d719 req-e85644eb-4b31-4b1b-b459-aaf602b54e2f service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Received unexpected event network-vif-plugged-5861aafc-4312-4d82-965c-2d1d993d8ea0 for instance with vm_state active and task_state shelving_image_uploading.
Apr 12 03:39:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:29 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Snapshot image upload complete
Apr 12 03:39:29 user nova-compute[72081]: DEBUG nova.compute.manager [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:39:29 user nova-compute[72081]: INFO nova.compute.manager [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Shelve offloading
Apr 12 03:39:29 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Instance destroyed successfully.
Apr 12 03:39:29 user nova-compute[72081]: DEBUG nova.compute.manager [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:39:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "refresh_cache-06481254-722a-4053-a6af-11f5dcc68636" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:39:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquired lock "refresh_cache-06481254-722a-4053-a6af-11f5dcc68636" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:39:29 user nova-compute[72081]: DEBUG nova.network.neutron [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:39:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:29 user nova-compute[72081]: DEBUG nova.network.neutron [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Updating instance_info_cache with network_info: [{"id": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "address": "fa:16:3e:c7:7c:eb", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.227", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5861aafc-43", "ovs_interfaceid": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:39:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Releasing lock "refresh_cache-06481254-722a-4053-a6af-11f5dcc68636" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:39:30 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Instance destroyed successfully.
Apr 12 03:39:30 user nova-compute[72081]: DEBUG nova.objects.instance [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lazy-loading 'resources' on Instance uuid 06481254-722a-4053-a6af-11f5dcc68636 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:39:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:37:30Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-405065337',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-405065337',id=20,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEV0Ykds0yE0AtGhC79nOfa8CiixAX+fWYFS4r0KbDrdX/eto3bHX8KjkQpgbhQK4OVD9q33q1+I1pg47ZtrC8YTjrFvz5r71kp0Qd9KtUm3DHiDVGrOt+n/RWN+EIIG+w==',key_name='tempest-keypair-2125155953',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:37:37Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='738ebb6b798c4974aa46f9bbe3828279',ramdisk_id='',reservation_id='r-jbxvhjdl',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-2139975165',owner_user_name='tempest-AttachVolumeShelveTestJSON-2139975165-project-member',shelved_at='2023-04-12T03:39:29.211136',shelved_host='user',shelved_image_id='291c6f89-241c-4edc-a92d-3497738ac9d3'},tags=<?>,task_state='shelving_offloading',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:39:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='de1fe5db01624224ab7b615c908d990b',uuid=06481254-722a-4053-a6af-11f5dcc68636,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='shelved') vif={"id": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "address": "fa:16:3e:c7:7c:eb", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.227", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5861aafc-43", "ovs_interfaceid": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:39:30 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Converting VIF {"id": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "address": "fa:16:3e:c7:7c:eb", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.227", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5861aafc-43", "ovs_interfaceid": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:39:30 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:c7:7c:eb,bridge_name='br-int',has_traffic_filtering=True,id=5861aafc-4312-4d82-965c-2d1d993d8ea0,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5861aafc-43') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:39:30 user nova-compute[72081]: DEBUG os_vif [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:c7:7c:eb,bridge_name='br-int',has_traffic_filtering=True,id=5861aafc-4312-4d82-965c-2d1d993d8ea0,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5861aafc-43') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:39:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap5861aafc-43, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:39:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-bce74e00-6642-4f96-9b99-cfef13e3b029 req-455425bc-49a0-4814-85f2-f840c2da980f service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Received event network-changed-5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:39:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-bce74e00-6642-4f96-9b99-cfef13e3b029 req-455425bc-49a0-4814-85f2-f840c2da980f service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Refreshing instance network info cache due to event network-changed-5861aafc-4312-4d82-965c-2d1d993d8ea0. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:39:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bce74e00-6642-4f96-9b99-cfef13e3b029 req-455425bc-49a0-4814-85f2-f840c2da980f service nova] Acquiring lock "refresh_cache-06481254-722a-4053-a6af-11f5dcc68636" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:39:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bce74e00-6642-4f96-9b99-cfef13e3b029 req-455425bc-49a0-4814-85f2-f840c2da980f service nova] Acquired lock "refresh_cache-06481254-722a-4053-a6af-11f5dcc68636" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:39:30 user nova-compute[72081]: DEBUG nova.network.neutron [req-bce74e00-6642-4f96-9b99-cfef13e3b029 req-455425bc-49a0-4814-85f2-f840c2da980f service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Refreshing network info cache for port 5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:39:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:39:30 user nova-compute[72081]: INFO os_vif [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:c7:7c:eb,bridge_name='br-int',has_traffic_filtering=True,id=5861aafc-4312-4d82-965c-2d1d993d8ea0,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5861aafc-43')
Apr 12 03:39:30 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Deleting instance files /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636_del
Apr 12 03:39:30 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Deletion of /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636_del complete
Apr 12 03:39:30 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Deleted allocations for instance 06481254-722a-4053-a6af-11f5dcc68636
Apr 12 03:39:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:39:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:39:30 user nova-compute[72081]: DEBUG nova.objects.instance [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lazy-loading 'numa_topology' on Instance uuid 06481254-722a-4053-a6af-11f5dcc68636 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:39:30 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:39:31 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:39:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.329s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:39:31 user nova-compute[72081]: DEBUG nova.network.neutron [req-bce74e00-6642-4f96-9b99-cfef13e3b029 req-455425bc-49a0-4814-85f2-f840c2da980f service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Updated VIF entry in instance network info cache for port 5861aafc-4312-4d82-965c-2d1d993d8ea0. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:39:31 user nova-compute[72081]: DEBUG nova.network.neutron [req-bce74e00-6642-4f96-9b99-cfef13e3b029 req-455425bc-49a0-4814-85f2-f840c2da980f service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Updating instance_info_cache with network_info: [{"id": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "address": "fa:16:3e:c7:7c:eb", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": null, "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.227", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "unbound", "details": {}, "devname": "tap5861aafc-43", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:39:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bce74e00-6642-4f96-9b99-cfef13e3b029 req-455425bc-49a0-4814-85f2-f840c2da980f service nova] Releasing lock "refresh_cache-06481254-722a-4053-a6af-11f5dcc68636" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:39:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-cfc43321-829f-4bc7-a28f-41c31d44851e tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "06481254-722a-4053-a6af-11f5dcc68636" "released" by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" :: held 5.882s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:39:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:39 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681270764.4020905, 23367b24-0a3b-454d-901d-8a3e5396333a => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:39:39 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] VM Stopped (Lifecycle Event)
Apr 12 03:39:39 user nova-compute[72081]: DEBUG nova.compute.manager [None req-879cc2a9-5f9f-47d4-af00-e4d958ba7387 None None] [instance: 23367b24-0a3b-454d-901d-8a3e5396333a] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:39:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:41 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681270766.0563555, 06481254-722a-4053-a6af-11f5dcc68636 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:39:41 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 06481254-722a-4053-a6af-11f5dcc68636] VM Stopped (Lifecycle Event)
Apr 12 03:39:41 user nova-compute[72081]: DEBUG nova.compute.manager [None req-d0639517-0a4a-41a1-a080-465d3d1f85b6 None None] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:39:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:45 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "d211dca2-af04-4dd3-b747-cc3bbd8d310b" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:39:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "d211dca2-af04-4dd3-b747-cc3bbd8d310b" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:39:47 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:39:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:39:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:39:47 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:39:47 user nova-compute[72081]: INFO nova.compute.claims [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Claim successful on node user
Apr 12 03:39:47 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:39:47 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:39:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.397s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:39:47 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:39:47 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:39:47 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:39:47 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:39:47 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:39:47 user nova-compute[72081]: INFO nova.virt.block_device [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Booting with volume snapshot 71d39e64-2f0b-4330-a785-fa95db0cc5d1 at /dev/vda
Apr 12 03:39:47 user nova-compute[72081]: DEBUG nova.policy [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'b00b348933224558aad1cb12921f100f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '003688353e2f4b36831c574ebaf65173', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:39:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:48 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Successfully created port: db8f85d3-266d-4a25-b809-3f4cf95a54d7 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:39:49 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Successfully updated port: db8f85d3-266d-4a25-b809-3f4cf95a54d7 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:39:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "refresh_cache-d211dca2-af04-4dd3-b747-cc3bbd8d310b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:39:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquired lock "refresh_cache-d211dca2-af04-4dd3-b747-cc3bbd8d310b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:39:49 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:39:49 user nova-compute[72081]: DEBUG nova.compute.manager [req-3f19dc39-328c-40bd-9483-735fcf6da550 req-56ddee45-cba9-4fd1-92d9-74e5e208933c service nova] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Received event network-changed-db8f85d3-266d-4a25-b809-3f4cf95a54d7 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:39:49 user nova-compute[72081]: DEBUG nova.compute.manager [req-3f19dc39-328c-40bd-9483-735fcf6da550 req-56ddee45-cba9-4fd1-92d9-74e5e208933c service nova] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Refreshing instance network info cache due to event network-changed-db8f85d3-266d-4a25-b809-3f4cf95a54d7. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:39:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3f19dc39-328c-40bd-9483-735fcf6da550 req-56ddee45-cba9-4fd1-92d9-74e5e208933c service nova] Acquiring lock "refresh_cache-d211dca2-af04-4dd3-b747-cc3bbd8d310b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:39:49 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:39:49 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Updating instance_info_cache with network_info: [{"id": "db8f85d3-266d-4a25-b809-3f4cf95a54d7", "address": "fa:16:3e:00:ac:f8", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdb8f85d3-26", "ovs_interfaceid": "db8f85d3-266d-4a25-b809-3f4cf95a54d7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:39:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Releasing lock "refresh_cache-d211dca2-af04-4dd3-b747-cc3bbd8d310b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:39:49 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Instance network_info: |[{"id": "db8f85d3-266d-4a25-b809-3f4cf95a54d7", "address": "fa:16:3e:00:ac:f8", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdb8f85d3-26", "ovs_interfaceid": "db8f85d3-266d-4a25-b809-3f4cf95a54d7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:39:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3f19dc39-328c-40bd-9483-735fcf6da550 req-56ddee45-cba9-4fd1-92d9-74e5e208933c service nova] Acquired lock "refresh_cache-d211dca2-af04-4dd3-b747-cc3bbd8d310b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:39:49 user nova-compute[72081]: DEBUG nova.network.neutron [req-3f19dc39-328c-40bd-9483-735fcf6da550 req-56ddee45-cba9-4fd1-92d9-74e5e208933c service nova] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Refreshing network info cache for port db8f85d3-266d-4a25-b809-3f4cf95a54d7 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:39:50 user nova-compute[72081]: DEBUG nova.network.neutron [req-3f19dc39-328c-40bd-9483-735fcf6da550 req-56ddee45-cba9-4fd1-92d9-74e5e208933c service nova] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Updated VIF entry in instance network info cache for port db8f85d3-266d-4a25-b809-3f4cf95a54d7. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:39:50 user nova-compute[72081]: DEBUG nova.network.neutron [req-3f19dc39-328c-40bd-9483-735fcf6da550 req-56ddee45-cba9-4fd1-92d9-74e5e208933c service nova] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Updating instance_info_cache with network_info: [{"id": "db8f85d3-266d-4a25-b809-3f4cf95a54d7", "address": "fa:16:3e:00:ac:f8", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdb8f85d3-26", "ovs_interfaceid": "db8f85d3-266d-4a25-b809-3f4cf95a54d7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:39:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3f19dc39-328c-40bd-9483-735fcf6da550 req-56ddee45-cba9-4fd1-92d9-74e5e208933c service nova] Releasing lock "refresh_cache-d211dca2-af04-4dd3-b747-cc3bbd8d310b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:39:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:53 user nova-compute[72081]: DEBUG os_brick.utils [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:39:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7b44821d-4c36-4a86-8bf6-669ead0ca770]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:39:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0baf3d32-9036-49d6-a937-89499cd4b862]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:39:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[02d73796-dc73-466b-8614-30f779d3d7ba]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:39:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c8f82e4b-a974-4243-a391-7e7ec482a4d3]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:39:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:39:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:39:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:39:53 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:39:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3eac912e-d98d-443f-b13c-847fb8138fa2]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:39:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:39:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:39:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:39:53 user nova-compute[72081]: DEBUG os_brick.utils [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] <== get_connector_properties: return (118ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:39:53 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Updating existing volume attachment record: 99a9c0b4-9a40-497c-bcee-197906bb0f3d {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:39:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:39:58 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Creating image(s)
Apr 12 03:39:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Did not create local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Ensure instance console log exists: /opt/stack/data/nova/instances/d211dca2-af04-4dd3-b747-cc3bbd8d310b/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Start _get_guest_xml network_info=[{"id": "db8f85d3-266d-4a25-b809-3f4cf95a54d7", "address": "fa:16:3e:00:ac:f8", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdb8f85d3-26", "ovs_interfaceid": "db8f85d3-266d-4a25-b809-3f4cf95a54d7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='d41d8cd98f00b204e9800998ecf8427e',container_format='bare',created_at=2023-04-12T03:39:45Z,direct_url=<?>,disk_format='qcow2',id=1ec76e72-2cb8-4d32-80d6-c41e19d43159,min_disk=1,min_ram=0,name='tempest-TestVolumeBootPatternsnapshot-1831025023',owner='003688353e2f4b36831c574ebaf65173',properties=ImageMetaProps,protected=<?>,size=0,status='active',tags=<?>,updated_at=2023-04-12T03:39:46Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HNXGMAWw', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'd211dca2-af04-4dd3-b747-cc3bbd8d310b', 'attached_at': '', 'detached_at': '', 'volume_id': '43533501-0b6e-4f4e-a56e-53944664ffff', 'serial': '43533501-0b6e-4f4e-a56e-53944664ffff'}, 'disk_bus': 'virtio', 'delete_on_termination': True, 'boot_index': 0, 'guest_format': None, 'device_type': 'disk', 'attachment_id': '99a9c0b4-9a40-497c-bcee-197906bb0f3d', 'volume_type': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:39:58 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:39:58 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:39:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='d41d8cd98f00b204e9800998ecf8427e',container_format='bare',created_at=2023-04-12T03:39:45Z,direct_url=<?>,disk_format='qcow2',id=1ec76e72-2cb8-4d32-80d6-c41e19d43159,min_disk=1,min_ram=0,name='tempest-TestVolumeBootPatternsnapshot-1831025023',owner='003688353e2f4b36831c574ebaf65173',properties=ImageMetaProps,protected=<?>,size=0,status='active',tags=<?>,updated_at=2023-04-12T03:39:46Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HNXGMAWw', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:39:58 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:39:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:39:58 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:39:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:39:58 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f8d94f21-5c34-48cf-b823-09131fcf785a]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:39:58 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0bf0b03b-78e5-4fa3-8732-ca5b6e710650]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff] added
Apr 12 03:39:58 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[583c66d5-8520-4369-b121-a71c302bc2f9]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3b473986-8d6d-44c9-bd68-ef28849aa32e]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff -p 172.16.0.220:3260 --op update -n node.session.auth.username -v HNXGMAWw {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff -p 172.16.0.220:3260 --op update -n node.session.auth.username -v HNXGMAWw" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a75f66c-43b2-4b9e-9d32-6f8a670dcd82]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'HNXGMAWw'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8938bb5a-3951-4fe2-b820-5fc98ca94f38]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[984b88d4-b04c-4aa8-81fc-aa8c82a93ba8]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:39:58 user nova-compute[72081]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)
Apr 12 03:39:58 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:39:58 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:39:58 user nova-compute[72081]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)
Apr 12 03:39:58 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:39:58 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff -p 172.16.0.220:3260 --login" returned: 0 in 0.065s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f956fee3-7077-42a3-9972-95f02f3239e3]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff, portal: 172.16.0.220,3260]
Apr 12 03:39:58 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff, portal: 172.16.0.220,3260] successful.
Apr 12 03:39:58 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e2e8922e-6d37-45ac-9d7d-b30c54727cc1]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b8565cf5-9b16-45d3-b81f-0b821dafc5f7]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:39:58 user nova-compute[72081]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)
Apr 12 03:39:58 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:39:58 user nova-compute[72081]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff (non-flash)
Apr 12 03:39:58 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:39:58 user nova-compute[72081]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)
Apr 12 03:39:58 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:39:58 user nova-compute[72081]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff (non-flash)
Apr 12 03:39:58 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.283s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] HCTL ('34', '-', '-', 0) found on session 26 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[910a9371-0f89-42fd-aaaa-ec3b35c318ce]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:39:58 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Searching for a device in session 26 and hctl ['34', '*', '*', 0] yield: None {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Searching for a device in session 26 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HNXGMAWw', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.312s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] <== connect_volume: return (1315ms) {'type': 'block', 'scsi_wwn': '23862636538663437', 'path': '/dev/sdc'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23862636538663437', 'path': '/dev/sdc'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:39:46Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-image-snapshot-server-759861809',display_name='tempest-TestVolumeBootPattern-image-snapshot-server-759861809',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-image-snapshot-server-759861809',id=26,image_ref='1ec76e72-2cb8-4d32-80d6-c41e19d43159',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='003688353e2f4b36831c574ebaf65173',ramdisk_id='',reservation_id='r-ooscgxt6',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_bdm_v2='True',image_boot_roles='reader,member',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_project_name='tempest-TestVolumeBootPattern-235043461',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestVolumeBootPattern-235043461-project-member',image_root_device_name='/dev/vda',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-235043461',owner_user_name='tempest-TestVolumeBootPattern-235043461-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:39:48Z,user_data=None,user_id='b00b348933224558aad1cb12921f100f',uuid=d211dca2-af04-4dd3-b747-cc3bbd8d310b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "db8f85d3-266d-4a25-b809-3f4cf95a54d7", "address": "fa:16:3e:00:ac:f8", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdb8f85d3-26", "ovs_interfaceid": "db8f85d3-266d-4a25-b809-3f4cf95a54d7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converting VIF {"id": "db8f85d3-266d-4a25-b809-3f4cf95a54d7", "address": "fa:16:3e:00:ac:f8", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdb8f85d3-26", "ovs_interfaceid": "db8f85d3-266d-4a25-b809-3f4cf95a54d7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:00:ac:f8,bridge_name='br-int',has_traffic_filtering=True,id=db8f85d3-266d-4a25-b809-3f4cf95a54d7,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdb8f85d3-26') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG nova.objects.instance [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lazy-loading 'pci_devices' on Instance uuid d211dca2-af04-4dd3-b747-cc3bbd8d310b {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:39:59 user nova-compute[72081]:   <uuid>d211dca2-af04-4dd3-b747-cc3bbd8d310b</uuid>
Apr 12 03:39:59 user nova-compute[72081]:   <name>instance-0000001a</name>
Apr 12 03:39:59 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:39:59 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:39:59 user nova-compute[72081]:   <metadata>
Apr 12 03:39:59 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:39:59 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:39:59 user nova-compute[72081]:       <nova:name>tempest-TestVolumeBootPattern-image-snapshot-server-759861809</nova:name>
Apr 12 03:39:59 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:39:58</nova:creationTime>
Apr 12 03:39:59 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:39:59 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:39:59 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:39:59 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:39:59 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:39:59 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:39:59 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:39:59 user nova-compute[72081]:       <nova:owner>
Apr 12 03:39:59 user nova-compute[72081]:         <nova:user uuid="b00b348933224558aad1cb12921f100f">tempest-TestVolumeBootPattern-235043461-project-member</nova:user>
Apr 12 03:39:59 user nova-compute[72081]:         <nova:project uuid="003688353e2f4b36831c574ebaf65173">tempest-TestVolumeBootPattern-235043461</nova:project>
Apr 12 03:39:59 user nova-compute[72081]:       </nova:owner>
Apr 12 03:39:59 user nova-compute[72081]:       <nova:root type="image" uuid="1ec76e72-2cb8-4d32-80d6-c41e19d43159"/>
Apr 12 03:39:59 user nova-compute[72081]:       <nova:ports>
Apr 12 03:39:59 user nova-compute[72081]:         <nova:port uuid="db8f85d3-266d-4a25-b809-3f4cf95a54d7">
Apr 12 03:39:59 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Apr 12 03:39:59 user nova-compute[72081]:         </nova:port>
Apr 12 03:39:59 user nova-compute[72081]:       </nova:ports>
Apr 12 03:39:59 user nova-compute[72081]:     </nova:instance>
Apr 12 03:39:59 user nova-compute[72081]:   </metadata>
Apr 12 03:39:59 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:39:59 user nova-compute[72081]:     <system>
Apr 12 03:39:59 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:39:59 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:39:59 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:39:59 user nova-compute[72081]:       <entry name="serial">d211dca2-af04-4dd3-b747-cc3bbd8d310b</entry>
Apr 12 03:39:59 user nova-compute[72081]:       <entry name="uuid">d211dca2-af04-4dd3-b747-cc3bbd8d310b</entry>
Apr 12 03:39:59 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:39:59 user nova-compute[72081]:     </system>
Apr 12 03:39:59 user nova-compute[72081]:   </sysinfo>
Apr 12 03:39:59 user nova-compute[72081]:   <os>
Apr 12 03:39:59 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:39:59 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:39:59 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:39:59 user nova-compute[72081]:   </os>
Apr 12 03:39:59 user nova-compute[72081]:   <features>
Apr 12 03:39:59 user nova-compute[72081]:     <acpi/>
Apr 12 03:39:59 user nova-compute[72081]:     <apic/>
Apr 12 03:39:59 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:39:59 user nova-compute[72081]:   </features>
Apr 12 03:39:59 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:39:59 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:39:59 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:39:59 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:39:59 user nova-compute[72081]:   </clock>
Apr 12 03:39:59 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:39:59 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:39:59 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:39:59 user nova-compute[72081]:   </cpu>
Apr 12 03:39:59 user nova-compute[72081]:   <devices>
Apr 12 03:39:59 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:39:59 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:39:59 user nova-compute[72081]:       <source dev="/dev/sdc"/>
Apr 12 03:39:59 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:39:59 user nova-compute[72081]:       <serial>43533501-0b6e-4f4e-a56e-53944664ffff</serial>
Apr 12 03:39:59 user nova-compute[72081]:     </disk>
Apr 12 03:39:59 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:39:59 user nova-compute[72081]:       <mac address="fa:16:3e:00:ac:f8"/>
Apr 12 03:39:59 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:39:59 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:39:59 user nova-compute[72081]:       <target dev="tapdb8f85d3-26"/>
Apr 12 03:39:59 user nova-compute[72081]:     </interface>
Apr 12 03:39:59 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:39:59 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/d211dca2-af04-4dd3-b747-cc3bbd8d310b/console.log" append="off"/>
Apr 12 03:39:59 user nova-compute[72081]:     </serial>
Apr 12 03:39:59 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:39:59 user nova-compute[72081]:     <video>
Apr 12 03:39:59 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:39:59 user nova-compute[72081]:     </video>
Apr 12 03:39:59 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:39:59 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:39:59 user nova-compute[72081]:     </rng>
Apr 12 03:39:59 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:39:59 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:39:59 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:39:59 user nova-compute[72081]:     </memballoon>
Apr 12 03:39:59 user nova-compute[72081]:   </devices>
Apr 12 03:39:59 user nova-compute[72081]: </domain>
Apr 12 03:39:59 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:39:46Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-image-snapshot-server-759861809',display_name='tempest-TestVolumeBootPattern-image-snapshot-server-759861809',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-image-snapshot-server-759861809',id=26,image_ref='1ec76e72-2cb8-4d32-80d6-c41e19d43159',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='003688353e2f4b36831c574ebaf65173',ramdisk_id='',reservation_id='r-ooscgxt6',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_bdm_v2='True',image_boot_roles='reader,member',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_project_name='tempest-TestVolumeBootPattern-235043461',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestVolumeBootPattern-235043461-project-member',image_root_device_name='/dev/vda',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-235043461',owner_user_name='tempest-TestVolumeBootPattern-235043461-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:39:48Z,user_data=None,user_id='b00b348933224558aad1cb12921f100f',uuid=d211dca2-af04-4dd3-b747-cc3bbd8d310b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "db8f85d3-266d-4a25-b809-3f4cf95a54d7", "address": "fa:16:3e:00:ac:f8", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdb8f85d3-26", "ovs_interfaceid": "db8f85d3-266d-4a25-b809-3f4cf95a54d7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converting VIF {"id": "db8f85d3-266d-4a25-b809-3f4cf95a54d7", "address": "fa:16:3e:00:ac:f8", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdb8f85d3-26", "ovs_interfaceid": "db8f85d3-266d-4a25-b809-3f4cf95a54d7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:00:ac:f8,bridge_name='br-int',has_traffic_filtering=True,id=db8f85d3-266d-4a25-b809-3f4cf95a54d7,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdb8f85d3-26') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG os_vif [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:00:ac:f8,bridge_name='br-int',has_traffic_filtering=True,id=db8f85d3-266d-4a25-b809-3f4cf95a54d7,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdb8f85d3-26') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapdb8f85d3-26, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapdb8f85d3-26, col_values=(('external_ids', {'iface-id': 'db8f85d3-266d-4a25-b809-3f4cf95a54d7', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:00:ac:f8', 'vm-uuid': 'd211dca2-af04-4dd3-b747-cc3bbd8d310b'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:39:59 user nova-compute[72081]: INFO os_vif [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:00:ac:f8,bridge_name='br-int',has_traffic_filtering=True,id=db8f85d3-266d-4a25-b809-3f4cf95a54d7,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdb8f85d3-26')
Apr 12 03:39:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] No VIF found with MAC fa:16:3e:00:ac:f8, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-31208b88-a3f2-4775-883d-ef085ef04126 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquiring lock "ff5a7005-6dd4-491a-a55c-db7a82dc28f7" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-31208b88-a3f2-4775-883d-ef085ef04126 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "ff5a7005-6dd4-491a-a55c-db7a82dc28f7" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-31208b88-a3f2-4775-883d-ef085ef04126 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquiring lock "ff5a7005-6dd4-491a-a55c-db7a82dc28f7-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-31208b88-a3f2-4775-883d-ef085ef04126 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "ff5a7005-6dd4-491a-a55c-db7a82dc28f7-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-31208b88-a3f2-4775-883d-ef085ef04126 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "ff5a7005-6dd4-491a-a55c-db7a82dc28f7-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:39:59 user nova-compute[72081]: INFO nova.compute.manager [None req-31208b88-a3f2-4775-883d-ef085ef04126 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Terminating instance
Apr 12 03:39:59 user nova-compute[72081]: DEBUG nova.compute.manager [None req-31208b88-a3f2-4775-883d-ef085ef04126 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:39:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:00 user nova-compute[72081]: DEBUG nova.compute.manager [req-a77b283b-b583-4537-9456-4b78291cab84 req-b98f3554-e9ab-4953-a67b-3ab3cf5aea71 service nova] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Received event network-vif-unplugged-a460c7a2-ed31-445e-a2a8-296e5163084b {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:40:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a77b283b-b583-4537-9456-4b78291cab84 req-b98f3554-e9ab-4953-a67b-3ab3cf5aea71 service nova] Acquiring lock "ff5a7005-6dd4-491a-a55c-db7a82dc28f7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a77b283b-b583-4537-9456-4b78291cab84 req-b98f3554-e9ab-4953-a67b-3ab3cf5aea71 service nova] Lock "ff5a7005-6dd4-491a-a55c-db7a82dc28f7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a77b283b-b583-4537-9456-4b78291cab84 req-b98f3554-e9ab-4953-a67b-3ab3cf5aea71 service nova] Lock "ff5a7005-6dd4-491a-a55c-db7a82dc28f7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:00 user nova-compute[72081]: DEBUG nova.compute.manager [req-a77b283b-b583-4537-9456-4b78291cab84 req-b98f3554-e9ab-4953-a67b-3ab3cf5aea71 service nova] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] No waiting events found dispatching network-vif-unplugged-a460c7a2-ed31-445e-a2a8-296e5163084b {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:40:00 user nova-compute[72081]: DEBUG nova.compute.manager [req-a77b283b-b583-4537-9456-4b78291cab84 req-b98f3554-e9ab-4953-a67b-3ab3cf5aea71 service nova] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Received event network-vif-unplugged-a460c7a2-ed31-445e-a2a8-296e5163084b for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:40:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:00 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Instance destroyed successfully.
Apr 12 03:40:00 user nova-compute[72081]: DEBUG nova.objects.instance [None req-31208b88-a3f2-4775-883d-ef085ef04126 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lazy-loading 'resources' on Instance uuid ff5a7005-6dd4-491a-a55c-db7a82dc28f7 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:40:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-31208b88-a3f2-4775-883d-ef085ef04126 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:38:05Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-1560370789',display_name='tempest-VolumesAdminNegativeTest-server-1560370789',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-1560370789',id=22,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:38:15Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='f4cce206e2b94b03ade2d310373c5ca2',ramdisk_id='',reservation_id='r-h9pa9k9c',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesAdminNegativeTest-2105871059',owner_user_name='tempest-VolumesAdminNegativeTest-2105871059-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:38:15Z,user_data=None,user_id='e2f496e5fe694d46a9803f797743a7dc',uuid=ff5a7005-6dd4-491a-a55c-db7a82dc28f7,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "a460c7a2-ed31-445e-a2a8-296e5163084b", "address": "fa:16:3e:5a:90:ca", "network": {"id": "263f9508-8639-4289-9210-f9807acf6766", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-337713645-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f4cce206e2b94b03ade2d310373c5ca2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa460c7a2-ed", "ovs_interfaceid": "a460c7a2-ed31-445e-a2a8-296e5163084b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:40:00 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-31208b88-a3f2-4775-883d-ef085ef04126 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Converting VIF {"id": "a460c7a2-ed31-445e-a2a8-296e5163084b", "address": "fa:16:3e:5a:90:ca", "network": {"id": "263f9508-8639-4289-9210-f9807acf6766", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-337713645-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f4cce206e2b94b03ade2d310373c5ca2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa460c7a2-ed", "ovs_interfaceid": "a460c7a2-ed31-445e-a2a8-296e5163084b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:40:00 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-31208b88-a3f2-4775-883d-ef085ef04126 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:5a:90:ca,bridge_name='br-int',has_traffic_filtering=True,id=a460c7a2-ed31-445e-a2a8-296e5163084b,network=Network(263f9508-8639-4289-9210-f9807acf6766),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa460c7a2-ed') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:40:00 user nova-compute[72081]: DEBUG os_vif [None req-31208b88-a3f2-4775-883d-ef085ef04126 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:5a:90:ca,bridge_name='br-int',has_traffic_filtering=True,id=a460c7a2-ed31-445e-a2a8-296e5163084b,network=Network(263f9508-8639-4289-9210-f9807acf6766),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa460c7a2-ed') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:40:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapa460c7a2-ed, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:40:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:40:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:00 user nova-compute[72081]: INFO os_vif [None req-31208b88-a3f2-4775-883d-ef085ef04126 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:5a:90:ca,bridge_name='br-int',has_traffic_filtering=True,id=a460c7a2-ed31-445e-a2a8-296e5163084b,network=Network(263f9508-8639-4289-9210-f9807acf6766),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa460c7a2-ed')
Apr 12 03:40:00 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-31208b88-a3f2-4775-883d-ef085ef04126 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Deleting instance files /opt/stack/data/nova/instances/ff5a7005-6dd4-491a-a55c-db7a82dc28f7_del
Apr 12 03:40:00 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-31208b88-a3f2-4775-883d-ef085ef04126 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Deletion of /opt/stack/data/nova/instances/ff5a7005-6dd4-491a-a55c-db7a82dc28f7_del complete
Apr 12 03:40:00 user nova-compute[72081]: INFO nova.compute.manager [None req-31208b88-a3f2-4775-883d-ef085ef04126 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Took 0.94 seconds to destroy the instance on the hypervisor.
Apr 12 03:40:00 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-31208b88-a3f2-4775-883d-ef085ef04126 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:40:00 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:40:00 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:40:01 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:40:01 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Took 0.48 seconds to deallocate network for instance.
Apr 12 03:40:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:01 user nova-compute[72081]: DEBUG nova.compute.manager [req-ee775869-eab7-4056-a6e6-0165bae5b289 req-4baf9fe8-6384-49e0-967e-2b15a063c091 service nova] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Received event network-vif-deleted-a460c7a2-ed31-445e-a2a8-296e5163084b {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:40:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-31208b88-a3f2-4775-883d-ef085ef04126 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-31208b88-a3f2-4775-883d-ef085ef04126 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:01 user nova-compute[72081]: DEBUG nova.compute.manager [req-0bbd2885-b390-4b67-b995-01d6de6490a7 req-b81d41b6-15fe-4937-940f-d4e40abd23af service nova] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Received event network-vif-plugged-db8f85d3-266d-4a25-b809-3f4cf95a54d7 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:40:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0bbd2885-b390-4b67-b995-01d6de6490a7 req-b81d41b6-15fe-4937-940f-d4e40abd23af service nova] Acquiring lock "d211dca2-af04-4dd3-b747-cc3bbd8d310b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0bbd2885-b390-4b67-b995-01d6de6490a7 req-b81d41b6-15fe-4937-940f-d4e40abd23af service nova] Lock "d211dca2-af04-4dd3-b747-cc3bbd8d310b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0bbd2885-b390-4b67-b995-01d6de6490a7 req-b81d41b6-15fe-4937-940f-d4e40abd23af service nova] Lock "d211dca2-af04-4dd3-b747-cc3bbd8d310b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:01 user nova-compute[72081]: DEBUG nova.compute.manager [req-0bbd2885-b390-4b67-b995-01d6de6490a7 req-b81d41b6-15fe-4937-940f-d4e40abd23af service nova] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] No waiting events found dispatching network-vif-plugged-db8f85d3-266d-4a25-b809-3f4cf95a54d7 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:40:01 user nova-compute[72081]: WARNING nova.compute.manager [req-0bbd2885-b390-4b67-b995-01d6de6490a7 req-b81d41b6-15fe-4937-940f-d4e40abd23af service nova] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Received unexpected event network-vif-plugged-db8f85d3-266d-4a25-b809-3f4cf95a54d7 for instance with vm_state building and task_state spawning.
Apr 12 03:40:01 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-31208b88-a3f2-4775-883d-ef085ef04126 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:40:01 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-31208b88-a3f2-4775-883d-ef085ef04126 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:40:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-31208b88-a3f2-4775-883d-ef085ef04126 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.375s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:01 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-31208b88-a3f2-4775-883d-ef085ef04126 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Deleted allocations for instance ff5a7005-6dd4-491a-a55c-db7a82dc28f7
Apr 12 03:40:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-31208b88-a3f2-4775-883d-ef085ef04126 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "ff5a7005-6dd4-491a-a55c-db7a82dc28f7" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.019s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:02 user nova-compute[72081]: DEBUG nova.compute.manager [req-7facc66b-b6e1-45e9-af92-d38c769a5228 req-395dec4b-8abc-4a5b-a46a-3ed82ae2db79 service nova] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Received event network-vif-plugged-a460c7a2-ed31-445e-a2a8-296e5163084b {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:40:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7facc66b-b6e1-45e9-af92-d38c769a5228 req-395dec4b-8abc-4a5b-a46a-3ed82ae2db79 service nova] Acquiring lock "ff5a7005-6dd4-491a-a55c-db7a82dc28f7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7facc66b-b6e1-45e9-af92-d38c769a5228 req-395dec4b-8abc-4a5b-a46a-3ed82ae2db79 service nova] Lock "ff5a7005-6dd4-491a-a55c-db7a82dc28f7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7facc66b-b6e1-45e9-af92-d38c769a5228 req-395dec4b-8abc-4a5b-a46a-3ed82ae2db79 service nova] Lock "ff5a7005-6dd4-491a-a55c-db7a82dc28f7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:02 user nova-compute[72081]: DEBUG nova.compute.manager [req-7facc66b-b6e1-45e9-af92-d38c769a5228 req-395dec4b-8abc-4a5b-a46a-3ed82ae2db79 service nova] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] No waiting events found dispatching network-vif-plugged-a460c7a2-ed31-445e-a2a8-296e5163084b {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:40:02 user nova-compute[72081]: WARNING nova.compute.manager [req-7facc66b-b6e1-45e9-af92-d38c769a5228 req-395dec4b-8abc-4a5b-a46a-3ed82ae2db79 service nova] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Received unexpected event network-vif-plugged-a460c7a2-ed31-445e-a2a8-296e5163084b for instance with vm_state deleted and task_state None.
Apr 12 03:40:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:03 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270803.4170585, d211dca2-af04-4dd3-b747-cc3bbd8d310b => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:40:03 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] VM Resumed (Lifecycle Event)
Apr 12 03:40:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:40:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:40:03 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Instance spawned successfully.
Apr 12 03:40:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Attempting to register defaults for the following image properties: ['hw_input_bus', 'hw_pointer_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:40:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:40:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:40:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:40:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:40:03 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:40:03 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270803.420356, d211dca2-af04-4dd3-b747-cc3bbd8d310b => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:40:03 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] VM Started (Lifecycle Event)
Apr 12 03:40:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-d8d86a49-07b5-42ed-b6fd-6a7118f31c2a req-e45a53ca-f73d-438a-b1ed-9c72e5201d06 service nova] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Received event network-vif-plugged-db8f85d3-266d-4a25-b809-3f4cf95a54d7 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:40:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d8d86a49-07b5-42ed-b6fd-6a7118f31c2a req-e45a53ca-f73d-438a-b1ed-9c72e5201d06 service nova] Acquiring lock "d211dca2-af04-4dd3-b747-cc3bbd8d310b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d8d86a49-07b5-42ed-b6fd-6a7118f31c2a req-e45a53ca-f73d-438a-b1ed-9c72e5201d06 service nova] Lock "d211dca2-af04-4dd3-b747-cc3bbd8d310b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d8d86a49-07b5-42ed-b6fd-6a7118f31c2a req-e45a53ca-f73d-438a-b1ed-9c72e5201d06 service nova] Lock "d211dca2-af04-4dd3-b747-cc3bbd8d310b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-d8d86a49-07b5-42ed-b6fd-6a7118f31c2a req-e45a53ca-f73d-438a-b1ed-9c72e5201d06 service nova] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] No waiting events found dispatching network-vif-plugged-db8f85d3-266d-4a25-b809-3f4cf95a54d7 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:40:03 user nova-compute[72081]: WARNING nova.compute.manager [req-d8d86a49-07b5-42ed-b6fd-6a7118f31c2a req-e45a53ca-f73d-438a-b1ed-9c72e5201d06 service nova] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Received unexpected event network-vif-plugged-db8f85d3-266d-4a25-b809-3f4cf95a54d7 for instance with vm_state building and task_state spawning.
Apr 12 03:40:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:40:03 user nova-compute[72081]: INFO nova.compute.manager [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Took 5.46 seconds to spawn the instance on the hypervisor.
Apr 12 03:40:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:40:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:40:03 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:40:03 user nova-compute[72081]: INFO nova.compute.manager [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Took 16.39 seconds to build instance.
Apr 12 03:40:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b4194d18-7e93-455a-81de-9a4472bd8c61 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "d211dca2-af04-4dd3-b747-cc3bbd8d310b" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 16.486s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:03 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:40:03 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:40:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:03 user nova-compute[72081]: INFO nova.compute.manager [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Detaching volume 0ca0118b-5d36-40b7-bb39-806756144b3a
Apr 12 03:40:03 user nova-compute[72081]: INFO nova.virt.block_device [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Attempting to driver detach volume 0ca0118b-5d36-40b7-bb39-806756144b3a from mountpoint /dev/vdb
Apr 12 03:40:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Attempting to detach device vdb from instance 48977eb2-ce43-4577-a1cb-340e35703dbd from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 03:40:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:40:03 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:40:03 user nova-compute[72081]:   <source dev="/dev/sdd"/>
Apr 12 03:40:03 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:40:03 user nova-compute[72081]:   <serial>0ca0118b-5d36-40b7-bb39-806756144b3a</serial>
Apr 12 03:40:03 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Apr 12 03:40:03 user nova-compute[72081]: </disk>
Apr 12 03:40:03 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:40:03 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Successfully detached device vdb from instance 48977eb2-ce43-4577-a1cb-340e35703dbd from the persistent domain config.
Apr 12 03:40:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 48977eb2-ce43-4577-a1cb-340e35703dbd from the live domain config. {{(pid=72081) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 12 03:40:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:40:03 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:40:03 user nova-compute[72081]:   <source dev="/dev/sdd"/>
Apr 12 03:40:03 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:40:03 user nova-compute[72081]:   <serial>0ca0118b-5d36-40b7-bb39-806756144b3a</serial>
Apr 12 03:40:03 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Apr 12 03:40:03 user nova-compute[72081]: </disk>
Apr 12 03:40:03 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Received event <DeviceRemovedEvent: 1681270804.3697424, 48977eb2-ce43-4577-a1cb-340e35703dbd => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72081) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 48977eb2-ce43-4577-a1cb-340e35703dbd {{(pid=72081) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 12 03:40:04 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Successfully detached device vdb from instance 48977eb2-ce43-4577-a1cb-340e35703dbd from the live domain config.
Apr 12 03:40:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'aYQxDBlt', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:40:04 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:40:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d575fb8c-50e4-4b23-a40d-934508b48acf]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:40:04 user nova-compute[72081]: No targets found.
Apr 12 03:40:04 user nova-compute[72081]: iSNS:
Apr 12 03:40:04 user nova-compute[72081]: No targets found.
Apr 12 03:40:04 user nova-compute[72081]: STATIC:
Apr 12 03:40:04 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf
Apr 12 03:40:04 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:40:04 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:40:04 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:40:04 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:40:04 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:40:04 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff
Apr 12 03:40:04 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:40:04 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:40:04 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a
Apr 12 03:40:04 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:40:04 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:40:04 user nova-compute[72081]: FIRMWARE:
Apr 12 03:40:04 user nova-compute[72081]: No targets found.
Apr 12 03:40:04 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:40:04 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:40:04 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4a6f667e-3017-4617-a3b4-fe92f8622617]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4cd42d57-5c82-4d0d-a7a0-5164dadb8bfb]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:40:04 user nova-compute[72081]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)
Apr 12 03:40:04 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:40:04 user nova-compute[72081]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff (non-flash)
Apr 12 03:40:04 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:40:04 user nova-compute[72081]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a (non-flash)
Apr 12 03:40:04 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:40:04 user nova-compute[72081]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff (non-flash)
Apr 12 03:40:04 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3f942d51b0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a'): ({'sdd'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Removing single pathed devices sdd {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0361edbd-2e11-403c-ba50-92b790c98129]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[191fdd07-8388-45d3-a9c6-e3f1c0669c6f]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Flushing IO for device /dev/sdd {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.012s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[949735b9-ee2a-4033-942c-e1355a809a36]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.039s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[afcee1c1-ed53-4e84-9862-159b001d931b]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] SCSI volumes sdd have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0e1fc39e-b824-40f1-9d31-2540763292d6]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --logout" returned: 0 in 0.055s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d9e6ddf7-dab8-4cad-8183-0d6e93cbed45]: (4, ('Logging out of session [sid: 24, target: iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a, portal: 172.16.0.220,3260]\nLogout of [sid: 24, target: iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 24, target: iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a, portal: 172.16.0.220,3260]
Apr 12 03:40:04 user nova-compute[72081]: Logout of [sid: 24, target: iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a, portal: 172.16.0.220,3260] successful.
Apr 12 03:40:04 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ca0118b-5d36-40b7-bb39-806756144b3a -p 172.16.0.220:3260 --op delete" returned: 0 in 0.028s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c4a119f3-95ed-4cac-a6e6-6de60524fdc4]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.310s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] <== disconnect_volume: return (311ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:40:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.171s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.188s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bf03ba16-e423-4042-a8f8-3ca2d0055617/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bf03ba16-e423-4042-a8f8-3ca2d0055617/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bf03ba16-e423-4042-a8f8-3ca2d0055617/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:05 user nova-compute[72081]: DEBUG nova.objects.instance [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'flavor' on Instance uuid 48977eb2-ce43-4577-a1cb-340e35703dbd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:40:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b0f82ded-cbb4-431a-b301-262db6c1401a tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.974s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bf03ba16-e423-4042-a8f8-3ca2d0055617/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:40:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json" returned: 0 in 0.163s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:40:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:40:07 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:40:07 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:40:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=7844MB free_disk=26.078052520751953GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:40:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 2d0904c3-74e0-47bb-a832-758a7035b2f5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:40:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance bcd267d2-5b2c-4ed2-b634-0210fae94dfd actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:40:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 48977eb2-ce43-4577-a1cb-340e35703dbd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:40:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:40:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 1eef2c91-dfad-43aa-b193-25970db83b79 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:40:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance f73fa883-75f0-4593-a89c-d009c6d4ebf0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:40:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 3d4407f4-e078-4777-a110-cb3f922be2e8 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:40:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance bf03ba16-e423-4042-a8f8-3ca2d0055617 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:40:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance d211dca2-af04-4dd3-b747-cc3bbd8d310b actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:40:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:40:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=1664MB phys_disk=40GB used_disk=6GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:40:07 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:40:07 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:40:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:40:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.406s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:10 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:40:10 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:40:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:40:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Didn't find any instances for network info cache update. {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 12 03:40:10 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:40:10 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:40:10 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:40:10 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:40:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:40:11 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:40:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:15 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681270800.5333536, ff5a7005-6dd4-491a-a55c-db7a82dc28f7 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:40:15 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] VM Stopped (Lifecycle Event)
Apr 12 03:40:15 user nova-compute[72081]: DEBUG nova.compute.manager [None req-859cafc2-b94c-447a-926b-45fded133874 None None] [instance: ff5a7005-6dd4-491a-a55c-db7a82dc28f7] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:40:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:40:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:40:18 user nova-compute[72081]: INFO nova.compute.claims [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Claim successful on node user
Apr 12 03:40:18 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:40:18 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:40:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.380s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:40:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:40:18 user nova-compute[72081]: DEBUG nova.network.neutron [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:40:18 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:40:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:40:19 user nova-compute[72081]: DEBUG nova.compute.manager [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:40:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:40:19 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Creating image(s)
Apr 12 03:40:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "/opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "/opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "/opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.005s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "ceec718a3559ac0750129010fcadb4c577075d0b" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "ceec718a3559ac0750129010fcadb4c577075d0b" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:19 user nova-compute[72081]: DEBUG nova.policy [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '3f8024260bf14d9995e67cb0d1bc8ddc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05e1e25933cd42df98fb060b8d2701dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:40:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ceec718a3559ac0750129010fcadb4c577075d0b.part --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ceec718a3559ac0750129010fcadb4c577075d0b.part --force-share --output=json" returned: 0 in 0.140s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:19 user nova-compute[72081]: DEBUG nova.virt.images [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] e198a023-7244-4f4f-b524-67b3b8abbd94 was qcow2, converting to raw {{(pid=72081) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 12 03:40:19 user nova-compute[72081]: DEBUG nova.privsep.utils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72081) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 12 03:40:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/ceec718a3559ac0750129010fcadb4c577075d0b.part /opt/stack/data/nova/instances/_base/ceec718a3559ac0750129010fcadb4c577075d0b.converted {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/ceec718a3559ac0750129010fcadb4c577075d0b.part /opt/stack/data/nova/instances/_base/ceec718a3559ac0750129010fcadb4c577075d0b.converted" returned: 0 in 0.270s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ceec718a3559ac0750129010fcadb4c577075d0b.converted --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG nova.network.neutron [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Successfully created port: c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ceec718a3559ac0750129010fcadb4c577075d0b.converted --force-share --output=json" returned: 0 in 0.156s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "ceec718a3559ac0750129010fcadb4c577075d0b" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.059s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ceec718a3559ac0750129010fcadb4c577075d0b --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ceec718a3559ac0750129010fcadb4c577075d0b --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "ceec718a3559ac0750129010fcadb4c577075d0b" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "ceec718a3559ac0750129010fcadb4c577075d0b" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ceec718a3559ac0750129010fcadb4c577075d0b --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ceec718a3559ac0750129010fcadb4c577075d0b --force-share --output=json" returned: 0 in 0.130s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/ceec718a3559ac0750129010fcadb4c577075d0b,backing_fmt=raw /opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/ceec718a3559ac0750129010fcadb4c577075d0b,backing_fmt=raw /opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk 1073741824" returned: 0 in 0.060s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "ceec718a3559ac0750129010fcadb4c577075d0b" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.198s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ceec718a3559ac0750129010fcadb4c577075d0b --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ceec718a3559ac0750129010fcadb4c577075d0b --force-share --output=json" returned: 0 in 0.149s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Checking if we can resize image /opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Cannot resize image /opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG nova.objects.instance [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lazy-loading 'migration_context' on Instance uuid 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Ensure instance console log exists: /opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG nova.network.neutron [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Successfully updated port: c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "refresh_cache-2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquired lock "refresh_cache-2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG nova.network.neutron [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG nova.compute.manager [req-b9aa54d0-4f61-42c3-953a-3f26bfecb8a5 req-b4bdb9e2-d24a-4bd6-ba6f-b7c6cfc34617 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Received event network-changed-c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG nova.compute.manager [req-b9aa54d0-4f61-42c3-953a-3f26bfecb8a5 req-b4bdb9e2-d24a-4bd6-ba6f-b7c6cfc34617 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Refreshing instance network info cache due to event network-changed-c904583a-ea96-45c9-ad62-d5e0cc68031f. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b9aa54d0-4f61-42c3-953a-3f26bfecb8a5 req-b4bdb9e2-d24a-4bd6-ba6f-b7c6cfc34617 service nova] Acquiring lock "refresh_cache-2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:40:20 user nova-compute[72081]: DEBUG nova.network.neutron [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.network.neutron [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Updating instance_info_cache with network_info: [{"id": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "address": "fa:16:3e:84:c4:9d", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc904583a-ea", "ovs_interfaceid": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Releasing lock "refresh_cache-2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.compute.manager [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Instance network_info: |[{"id": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "address": "fa:16:3e:84:c4:9d", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc904583a-ea", "ovs_interfaceid": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b9aa54d0-4f61-42c3-953a-3f26bfecb8a5 req-b4bdb9e2-d24a-4bd6-ba6f-b7c6cfc34617 service nova] Acquired lock "refresh_cache-2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.network.neutron [req-b9aa54d0-4f61-42c3-953a-3f26bfecb8a5 req-b4bdb9e2-d24a-4bd6-ba6f-b7c6cfc34617 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Refreshing network info cache for port c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Start _get_guest_xml network_info=[{"id": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "address": "fa:16:3e:84:c4:9d", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc904583a-ea", "ovs_interfaceid": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:40:15Z,direct_url=<?>,disk_format='qcow2',id=e198a023-7244-4f4f-b524-67b3b8abbd94,min_disk=0,min_ram=0,name='tempest-scenario-img--1962517446',owner='05e1e25933cd42df98fb060b8d2701dc',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:40:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e198a023-7244-4f4f-b524-67b3b8abbd94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:40:21 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:40:21 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:40:15Z,direct_url=<?>,disk_format='qcow2',id=e198a023-7244-4f4f-b524-67b3b8abbd94,min_disk=0,min_ram=0,name='tempest-scenario-img--1962517446',owner='05e1e25933cd42df98fb060b8d2701dc',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:40:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:40:17Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-470315255',display_name='tempest-TestMinimumBasicScenario-server-470315255',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-470315255',id=27,image_ref='e198a023-7244-4f4f-b524-67b3b8abbd94',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHPcbdFPNwSacXUiuG3oK16SVNbJsO/MbG4r56gwQ5WdhiWutK0MgzDXL9ED179Ry++/GuDPfwLne43xpH8kd84JNZdPdBD50aHcS2/YXE5ZEXdAfa8QEldhEXCO3fgFsQ==',key_name='tempest-TestMinimumBasicScenario-1199406481',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='05e1e25933cd42df98fb060b8d2701dc',ramdisk_id='',reservation_id='r-220aqgyr',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e198a023-7244-4f4f-b524-67b3b8abbd94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-TestMinimumBasicScenario-1605347971',owner_user_name='tempest-TestMinimumBasicScenario-1605347971-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:40:19Z,user_data=None,user_id='3f8024260bf14d9995e67cb0d1bc8ddc',uuid=2d7f71ac-a671-4feb-8c2d-4cbcb84f7170,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "address": "fa:16:3e:84:c4:9d", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc904583a-ea", "ovs_interfaceid": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Converting VIF {"id": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "address": "fa:16:3e:84:c4:9d", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc904583a-ea", "ovs_interfaceid": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:84:c4:9d,bridge_name='br-int',has_traffic_filtering=True,id=c904583a-ea96-45c9-ad62-d5e0cc68031f,network=Network(91a7501a-44a5-4d48-a1f7-e1195ac99607),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc904583a-ea') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.objects.instance [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lazy-loading 'pci_devices' on Instance uuid 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:40:21 user nova-compute[72081]:   <uuid>2d7f71ac-a671-4feb-8c2d-4cbcb84f7170</uuid>
Apr 12 03:40:21 user nova-compute[72081]:   <name>instance-0000001b</name>
Apr 12 03:40:21 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:40:21 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:40:21 user nova-compute[72081]:   <metadata>
Apr 12 03:40:21 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:40:21 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:40:21 user nova-compute[72081]:       <nova:name>tempest-TestMinimumBasicScenario-server-470315255</nova:name>
Apr 12 03:40:21 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:40:21</nova:creationTime>
Apr 12 03:40:21 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:40:21 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:40:21 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:40:21 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:40:21 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:40:21 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:40:21 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:40:21 user nova-compute[72081]:       <nova:owner>
Apr 12 03:40:21 user nova-compute[72081]:         <nova:user uuid="3f8024260bf14d9995e67cb0d1bc8ddc">tempest-TestMinimumBasicScenario-1605347971-project-member</nova:user>
Apr 12 03:40:21 user nova-compute[72081]:         <nova:project uuid="05e1e25933cd42df98fb060b8d2701dc">tempest-TestMinimumBasicScenario-1605347971</nova:project>
Apr 12 03:40:21 user nova-compute[72081]:       </nova:owner>
Apr 12 03:40:21 user nova-compute[72081]:       <nova:root type="image" uuid="e198a023-7244-4f4f-b524-67b3b8abbd94"/>
Apr 12 03:40:21 user nova-compute[72081]:       <nova:ports>
Apr 12 03:40:21 user nova-compute[72081]:         <nova:port uuid="c904583a-ea96-45c9-ad62-d5e0cc68031f">
Apr 12 03:40:21 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.11" ipVersion="4"/>
Apr 12 03:40:21 user nova-compute[72081]:         </nova:port>
Apr 12 03:40:21 user nova-compute[72081]:       </nova:ports>
Apr 12 03:40:21 user nova-compute[72081]:     </nova:instance>
Apr 12 03:40:21 user nova-compute[72081]:   </metadata>
Apr 12 03:40:21 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:40:21 user nova-compute[72081]:     <system>
Apr 12 03:40:21 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:40:21 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:40:21 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:40:21 user nova-compute[72081]:       <entry name="serial">2d7f71ac-a671-4feb-8c2d-4cbcb84f7170</entry>
Apr 12 03:40:21 user nova-compute[72081]:       <entry name="uuid">2d7f71ac-a671-4feb-8c2d-4cbcb84f7170</entry>
Apr 12 03:40:21 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:40:21 user nova-compute[72081]:     </system>
Apr 12 03:40:21 user nova-compute[72081]:   </sysinfo>
Apr 12 03:40:21 user nova-compute[72081]:   <os>
Apr 12 03:40:21 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:40:21 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:40:21 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:40:21 user nova-compute[72081]:   </os>
Apr 12 03:40:21 user nova-compute[72081]:   <features>
Apr 12 03:40:21 user nova-compute[72081]:     <acpi/>
Apr 12 03:40:21 user nova-compute[72081]:     <apic/>
Apr 12 03:40:21 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:40:21 user nova-compute[72081]:   </features>
Apr 12 03:40:21 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:40:21 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:40:21 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:40:21 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:40:21 user nova-compute[72081]:   </clock>
Apr 12 03:40:21 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:40:21 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:40:21 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:40:21 user nova-compute[72081]:   </cpu>
Apr 12 03:40:21 user nova-compute[72081]:   <devices>
Apr 12 03:40:21 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:40:21 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:40:21 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk"/>
Apr 12 03:40:21 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:40:21 user nova-compute[72081]:     </disk>
Apr 12 03:40:21 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:40:21 user nova-compute[72081]:       <mac address="fa:16:3e:84:c4:9d"/>
Apr 12 03:40:21 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:40:21 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:40:21 user nova-compute[72081]:       <target dev="tapc904583a-ea"/>
Apr 12 03:40:21 user nova-compute[72081]:     </interface>
Apr 12 03:40:21 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:40:21 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/console.log" append="off"/>
Apr 12 03:40:21 user nova-compute[72081]:     </serial>
Apr 12 03:40:21 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:40:21 user nova-compute[72081]:     <video>
Apr 12 03:40:21 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:40:21 user nova-compute[72081]:     </video>
Apr 12 03:40:21 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:40:21 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:40:21 user nova-compute[72081]:     </rng>
Apr 12 03:40:21 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:40:21 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:40:21 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:40:21 user nova-compute[72081]:     </memballoon>
Apr 12 03:40:21 user nova-compute[72081]:   </devices>
Apr 12 03:40:21 user nova-compute[72081]: </domain>
Apr 12 03:40:21 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:40:17Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-470315255',display_name='tempest-TestMinimumBasicScenario-server-470315255',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-470315255',id=27,image_ref='e198a023-7244-4f4f-b524-67b3b8abbd94',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHPcbdFPNwSacXUiuG3oK16SVNbJsO/MbG4r56gwQ5WdhiWutK0MgzDXL9ED179Ry++/GuDPfwLne43xpH8kd84JNZdPdBD50aHcS2/YXE5ZEXdAfa8QEldhEXCO3fgFsQ==',key_name='tempest-TestMinimumBasicScenario-1199406481',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='05e1e25933cd42df98fb060b8d2701dc',ramdisk_id='',reservation_id='r-220aqgyr',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e198a023-7244-4f4f-b524-67b3b8abbd94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-TestMinimumBasicScenario-1605347971',owner_user_name='tempest-TestMinimumBasicScenario-1605347971-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:40:19Z,user_data=None,user_id='3f8024260bf14d9995e67cb0d1bc8ddc',uuid=2d7f71ac-a671-4feb-8c2d-4cbcb84f7170,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "address": "fa:16:3e:84:c4:9d", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc904583a-ea", "ovs_interfaceid": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Converting VIF {"id": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "address": "fa:16:3e:84:c4:9d", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc904583a-ea", "ovs_interfaceid": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:84:c4:9d,bridge_name='br-int',has_traffic_filtering=True,id=c904583a-ea96-45c9-ad62-d5e0cc68031f,network=Network(91a7501a-44a5-4d48-a1f7-e1195ac99607),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc904583a-ea') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG os_vif [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:84:c4:9d,bridge_name='br-int',has_traffic_filtering=True,id=c904583a-ea96-45c9-ad62-d5e0cc68031f,network=Network(91a7501a-44a5-4d48-a1f7-e1195ac99607),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc904583a-ea') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapc904583a-ea, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapc904583a-ea, col_values=(('external_ids', {'iface-id': 'c904583a-ea96-45c9-ad62-d5e0cc68031f', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:84:c4:9d', 'vm-uuid': '2d7f71ac-a671-4feb-8c2d-4cbcb84f7170'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:21 user nova-compute[72081]: INFO os_vif [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:84:c4:9d,bridge_name='br-int',has_traffic_filtering=True,id=c904583a-ea96-45c9-ad62-d5e0cc68031f,network=Network(91a7501a-44a5-4d48-a1f7-e1195ac99607),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc904583a-ea')
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] No VIF found with MAC fa:16:3e:84:c4:9d, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.network.neutron [req-b9aa54d0-4f61-42c3-953a-3f26bfecb8a5 req-b4bdb9e2-d24a-4bd6-ba6f-b7c6cfc34617 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Updated VIF entry in instance network info cache for port c904583a-ea96-45c9-ad62-d5e0cc68031f. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG nova.network.neutron [req-b9aa54d0-4f61-42c3-953a-3f26bfecb8a5 req-b4bdb9e2-d24a-4bd6-ba6f-b7c6cfc34617 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Updating instance_info_cache with network_info: [{"id": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "address": "fa:16:3e:84:c4:9d", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc904583a-ea", "ovs_interfaceid": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:40:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b9aa54d0-4f61-42c3-953a-3f26bfecb8a5 req-b4bdb9e2-d24a-4bd6-ba6f-b7c6cfc34617 service nova] Releasing lock "refresh_cache-2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:40:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:23 user nova-compute[72081]: DEBUG nova.compute.manager [req-290d2102-3226-481f-8c9b-dd9aa4be3b2c req-04b0a7c6-7ed7-4970-9949-76038fa2d515 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Received event network-vif-plugged-c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:40:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-290d2102-3226-481f-8c9b-dd9aa4be3b2c req-04b0a7c6-7ed7-4970-9949-76038fa2d515 service nova] Acquiring lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-290d2102-3226-481f-8c9b-dd9aa4be3b2c req-04b0a7c6-7ed7-4970-9949-76038fa2d515 service nova] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-290d2102-3226-481f-8c9b-dd9aa4be3b2c req-04b0a7c6-7ed7-4970-9949-76038fa2d515 service nova] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:23 user nova-compute[72081]: DEBUG nova.compute.manager [req-290d2102-3226-481f-8c9b-dd9aa4be3b2c req-04b0a7c6-7ed7-4970-9949-76038fa2d515 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] No waiting events found dispatching network-vif-plugged-c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:40:23 user nova-compute[72081]: WARNING nova.compute.manager [req-290d2102-3226-481f-8c9b-dd9aa4be3b2c req-04b0a7c6-7ed7-4970-9949-76038fa2d515 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Received unexpected event network-vif-plugged-c904583a-ea96-45c9-ad62-d5e0cc68031f for instance with vm_state building and task_state spawning.
Apr 12 03:40:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:25 user nova-compute[72081]: DEBUG nova.compute.manager [req-4bbbcd52-c86e-471b-b3d0-ef29d3f35e5a req-b52887e8-6341-40ef-9b9a-fb3bfa42e12b service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Received event network-vif-plugged-c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:40:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4bbbcd52-c86e-471b-b3d0-ef29d3f35e5a req-b52887e8-6341-40ef-9b9a-fb3bfa42e12b service nova] Acquiring lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4bbbcd52-c86e-471b-b3d0-ef29d3f35e5a req-b52887e8-6341-40ef-9b9a-fb3bfa42e12b service nova] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.003s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4bbbcd52-c86e-471b-b3d0-ef29d3f35e5a req-b52887e8-6341-40ef-9b9a-fb3bfa42e12b service nova] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:25 user nova-compute[72081]: DEBUG nova.compute.manager [req-4bbbcd52-c86e-471b-b3d0-ef29d3f35e5a req-b52887e8-6341-40ef-9b9a-fb3bfa42e12b service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] No waiting events found dispatching network-vif-plugged-c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:40:25 user nova-compute[72081]: WARNING nova.compute.manager [req-4bbbcd52-c86e-471b-b3d0-ef29d3f35e5a req-b52887e8-6341-40ef-9b9a-fb3bfa42e12b service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Received unexpected event network-vif-plugged-c904583a-ea96-45c9-ad62-d5e0cc68031f for instance with vm_state building and task_state spawning.
Apr 12 03:40:25 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270825.1496, 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:40:25 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] VM Resumed (Lifecycle Event)
Apr 12 03:40:25 user nova-compute[72081]: DEBUG nova.compute.manager [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:40:25 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:40:25 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Instance spawned successfully.
Apr 12 03:40:25 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:40:25 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:40:25 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:40:25 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:40:25 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:40:25 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:40:25 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:40:25 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:40:25 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:40:25 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:40:25 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270825.1504438, 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:40:25 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] VM Started (Lifecycle Event)
Apr 12 03:40:25 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:40:25 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:40:25 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:40:25 user nova-compute[72081]: INFO nova.compute.manager [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Took 6.30 seconds to spawn the instance on the hypervisor.
Apr 12 03:40:25 user nova-compute[72081]: DEBUG nova.compute.manager [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:40:25 user nova-compute[72081]: INFO nova.compute.manager [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Took 7.01 seconds to build instance.
Apr 12 03:40:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7d6d4fac-cdad-4d55-baf1-0de0081073d8 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.115s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:25 user nova-compute[72081]: DEBUG nova.compute.manager [req-1a66206a-9b2e-4b1e-93a2-350e2e2eadf1 req-f6a1a480-e34d-4ad1-88f1-e8ffd5799229 service nova] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Received event network-changed-979f4007-c0b3-408e-9f1c-a62c05af6417 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:40:25 user nova-compute[72081]: DEBUG nova.compute.manager [req-1a66206a-9b2e-4b1e-93a2-350e2e2eadf1 req-f6a1a480-e34d-4ad1-88f1-e8ffd5799229 service nova] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Refreshing instance network info cache due to event network-changed-979f4007-c0b3-408e-9f1c-a62c05af6417. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:40:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a66206a-9b2e-4b1e-93a2-350e2e2eadf1 req-f6a1a480-e34d-4ad1-88f1-e8ffd5799229 service nova] Acquiring lock "refresh_cache-f73fa883-75f0-4593-a89c-d009c6d4ebf0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:40:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a66206a-9b2e-4b1e-93a2-350e2e2eadf1 req-f6a1a480-e34d-4ad1-88f1-e8ffd5799229 service nova] Acquired lock "refresh_cache-f73fa883-75f0-4593-a89c-d009c6d4ebf0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:40:25 user nova-compute[72081]: DEBUG nova.network.neutron [req-1a66206a-9b2e-4b1e-93a2-350e2e2eadf1 req-f6a1a480-e34d-4ad1-88f1-e8ffd5799229 service nova] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Refreshing network info cache for port 979f4007-c0b3-408e-9f1c-a62c05af6417 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:40:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:26 user nova-compute[72081]: DEBUG nova.network.neutron [req-1a66206a-9b2e-4b1e-93a2-350e2e2eadf1 req-f6a1a480-e34d-4ad1-88f1-e8ffd5799229 service nova] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Updated VIF entry in instance network info cache for port 979f4007-c0b3-408e-9f1c-a62c05af6417. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:40:26 user nova-compute[72081]: DEBUG nova.network.neutron [req-1a66206a-9b2e-4b1e-93a2-350e2e2eadf1 req-f6a1a480-e34d-4ad1-88f1-e8ffd5799229 service nova] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Updating instance_info_cache with network_info: [{"id": "979f4007-c0b3-408e-9f1c-a62c05af6417", "address": "fa:16:3e:fd:fc:16", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.81", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap979f4007-c0", "ovs_interfaceid": "979f4007-c0b3-408e-9f1c-a62c05af6417", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:40:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a66206a-9b2e-4b1e-93a2-350e2e2eadf1 req-f6a1a480-e34d-4ad1-88f1-e8ffd5799229 service nova] Releasing lock "refresh_cache-f73fa883-75f0-4593-a89c-d009c6d4ebf0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:40:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG nova.objects.instance [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lazy-loading 'flavor' on Instance uuid f73fa883-75f0-4593-a89c-d009c6d4ebf0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.066s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:27 user nova-compute[72081]: INFO nova.compute.manager [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Attaching volume 492e66f2-98d6-43d6-bbe6-cbe61f9335f6 to /dev/vdb
Apr 12 03:40:27 user nova-compute[72081]: DEBUG os_brick.utils [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7945c02a-e6e3-4cd6-b93b-db2883833204]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9b4f6301-4a0f-435e-b948-9ade530024f2]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[87a31812-d605-4923-a091-e21479cee6f4]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[013eb7e4-fdbf-449b-a70c-c1248f1f79a2]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:40:27 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:40:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[39235d63-5325-43b3-8577-c91d82152579]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG os_brick.utils [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] <== get_connector_properties: return (136ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:40:27 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Updating existing volume attachment record: e4bcfe73-7297-4fb3-8f5a-c26b96874666 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:40:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'EMwRIslb', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:40:32 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:40:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:40:32 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:40:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:40:32 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4a7716b3-ffa8-4254-a4e1-d404b6f66426]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:40:32 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6e92d5e8-970d-45f8-b6bc-35519eb01bd9]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6] added
Apr 12 03:40:32 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6b92a26a-da3d-4623-bad2-610eabd1a948]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[789fa6bd-0dab-4b25-b86d-ea354dc5f5e0]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v EMwRIslb {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v EMwRIslb" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[01c9fd1a-71e9-439e-8244-a760211756fe]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'EMwRIslb'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[fbee9422-2ccd-4a77-ad19-97d40508b8c7]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[082feef6-b00a-432c-97aa-c23caf987ff5]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:40:32 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:40:32 user nova-compute[72081]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff (non-flash)
Apr 12 03:40:32 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:40:32 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:40:32 user nova-compute[72081]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff (non-flash)
Apr 12 03:40:32 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 -p 172.16.0.220:3260 --login" returned: 0 in 0.064s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[73b4b7a8-e27e-4da5-b3a7-f7c17f6994d2]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6, portal: 172.16.0.220,3260]
Apr 12 03:40:32 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6, portal: 172.16.0.220,3260] successful.
Apr 12 03:40:32 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2bc4fd2f-a860-4f4e-96d4-db5cd5416493]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d123d5c7-7d34-47ab-8c5e-2e7acf84edac]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:40:32 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:40:32 user nova-compute[72081]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff (non-flash)
Apr 12 03:40:32 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:40:32 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:40:32 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:40:32 user nova-compute[72081]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff (non-flash)
Apr 12 03:40:32 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:40:32 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.295s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] HCTL ('35', '-', '-', 0) found on session 27 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[57478bb4-fcc1-4d2a-87ac-1ffdd5457c35]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Searching for a device in session 27 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:40:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'EMwRIslb', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:40:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.331s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:40:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] <== connect_volume: return (1333ms) {'type': 'block', 'scsi_wwn': '23263313063666166', 'path': '/dev/sdd'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:40:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23263313063666166', 'path': '/dev/sdd'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:40:33 user nova-compute[72081]: DEBUG nova.objects.instance [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lazy-loading 'flavor' on Instance uuid f73fa883-75f0-4593-a89c-d009c6d4ebf0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:40:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] attach device xml: <disk type="block" device="disk">
Apr 12 03:40:33 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:40:33 user nova-compute[72081]:   <source dev="/dev/sdd"/>
Apr 12 03:40:33 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:40:33 user nova-compute[72081]:   <serial>492e66f2-98d6-43d6-bbe6-cbe61f9335f6</serial>
Apr 12 03:40:33 user nova-compute[72081]: </disk>
Apr 12 03:40:33 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 03:40:34 user nova-compute[72081]: DEBUG nova.compute.manager [None req-0d519d7a-4c9d-4493-884a-c37bc42b99f8 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:40:34 user nova-compute[72081]: INFO nova.compute.manager [None req-0d519d7a-4c9d-4493-884a-c37bc42b99f8 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] instance snapshotting
Apr 12 03:40:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:40:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:40:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] No VIF found with MAC fa:16:3e:fd:fc:16, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:40:34 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-0d519d7a-4c9d-4493-884a-c37bc42b99f8 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Beginning live snapshot process
Apr 12 03:40:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0d519d7a-4c9d-4493-884a-c37bc42b99f8 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json -f qcow2 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-64e003c7-95b9-46e3-912b-cffbafff7111 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.954s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0d519d7a-4c9d-4493-884a-c37bc42b99f8 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json -f qcow2" returned: 0 in 0.164s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0d519d7a-4c9d-4493-884a-c37bc42b99f8 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json -f qcow2 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0d519d7a-4c9d-4493-884a-c37bc42b99f8 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json -f qcow2" returned: 0 in 0.132s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0d519d7a-4c9d-4493-884a-c37bc42b99f8 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0d519d7a-4c9d-4493-884a-c37bc42b99f8 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.133s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0d519d7a-4c9d-4493-884a-c37bc42b99f8 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmpakux3id1/ed3fd39291a24e409a03189f36d78c5c.delta 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0d519d7a-4c9d-4493-884a-c37bc42b99f8 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmpakux3id1/ed3fd39291a24e409a03189f36d78c5c.delta 1073741824" returned: 0 in 0.067s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:35 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-0d519d7a-4c9d-4493-884a-c37bc42b99f8 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Quiescing instance not available: QEMU guest agent is not enabled.
Apr 12 03:40:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-0d519d7a-4c9d-4493-884a-c37bc42b99f8 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] COPY block job progress, current cursor: 0 final cursor: 43778048 {{(pid=72081) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 12 03:40:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-0d519d7a-4c9d-4493-884a-c37bc42b99f8 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] COPY block job progress, current cursor: 43778048 final cursor: 43778048 {{(pid=72081) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 12 03:40:36 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-0d519d7a-4c9d-4493-884a-c37bc42b99f8 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Skipping quiescing instance: QEMU guest agent is not enabled.
Apr 12 03:40:36 user nova-compute[72081]: DEBUG nova.privsep.utils [None req-0d519d7a-4c9d-4493-884a-c37bc42b99f8 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72081) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 12 03:40:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0d519d7a-4c9d-4493-884a-c37bc42b99f8 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmpakux3id1/ed3fd39291a24e409a03189f36d78c5c.delta /opt/stack/data/nova/instances/snapshots/tmpakux3id1/ed3fd39291a24e409a03189f36d78c5c {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-0d519d7a-4c9d-4493-884a-c37bc42b99f8 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmpakux3id1/ed3fd39291a24e409a03189f36d78c5c.delta /opt/stack/data/nova/instances/snapshots/tmpakux3id1/ed3fd39291a24e409a03189f36d78c5c" returned: 0 in 0.643s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:37 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-0d519d7a-4c9d-4493-884a-c37bc42b99f8 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Snapshot extracted, beginning image upload
Apr 12 03:40:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "117ea473-f8e3-43c7-8406-d4590422cc0b" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "117ea473-f8e3-43c7-8406-d4590422cc0b" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:40:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:40:38 user nova-compute[72081]: INFO nova.compute.claims [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Claim successful on node user
Apr 12 03:40:39 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:40:39 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:40:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.481s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:39 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:40:39 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:40:39 user nova-compute[72081]: DEBUG nova.network.neutron [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:40:39 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:40:39 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:40:39 user nova-compute[72081]: DEBUG nova.policy [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd52637cca2114e5e99e688ee371c835a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd13546149a4e44e1b782b99e9eb452e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:40:39 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:40:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:40:39 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Creating image(s)
Apr 12 03:40:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "/opt/stack/data/nova/instances/117ea473-f8e3-43c7-8406-d4590422cc0b/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "/opt/stack/data/nova/instances/117ea473-f8e3-43c7-8406-d4590422cc0b/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "/opt/stack/data/nova/instances/117ea473-f8e3-43c7-8406-d4590422cc0b/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:39 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-0d519d7a-4c9d-4493-884a-c37bc42b99f8 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Snapshot image upload complete
Apr 12 03:40:39 user nova-compute[72081]: INFO nova.compute.manager [None req-0d519d7a-4c9d-4493-884a-c37bc42b99f8 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Took 5.37 seconds to snapshot the instance on the hypervisor.
Apr 12 03:40:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.277s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.158s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/117ea473-f8e3-43c7-8406-d4590422cc0b/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/117ea473-f8e3-43c7-8406-d4590422cc0b/disk 1073741824" returned: 0 in 0.060s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.224s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.147s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:40 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Checking if we can resize image /opt/stack/data/nova/instances/117ea473-f8e3-43c7-8406-d4590422cc0b/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:40:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/117ea473-f8e3-43c7-8406-d4590422cc0b/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:40:40 user nova-compute[72081]: DEBUG nova.network.neutron [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Successfully created port: ec1c8d01-1c15-4a89-9205-23f04ec6fc58 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:40:40 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/117ea473-f8e3-43c7-8406-d4590422cc0b/disk --force-share --output=json" returned: 0 in 0.163s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:40:40 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Cannot resize image /opt/stack/data/nova/instances/117ea473-f8e3-43c7-8406-d4590422cc0b/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:40:40 user nova-compute[72081]: DEBUG nova.objects.instance [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lazy-loading 'migration_context' on Instance uuid 117ea473-f8e3-43c7-8406-d4590422cc0b {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:40:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:40:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Ensure instance console log exists: /opt/stack/data/nova/instances/117ea473-f8e3-43c7-8406-d4590422cc0b/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:40:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.network.neutron [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Successfully updated port: ec1c8d01-1c15-4a89-9205-23f04ec6fc58 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "refresh_cache-117ea473-f8e3-43c7-8406-d4590422cc0b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquired lock "refresh_cache-117ea473-f8e3-43c7-8406-d4590422cc0b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.network.neutron [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.compute.manager [req-d910653b-2432-4848-a5b1-55b1a74025e6 req-e358838f-2a49-461f-8c1b-338321c5d2a7 service nova] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Received event network-changed-ec1c8d01-1c15-4a89-9205-23f04ec6fc58 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.compute.manager [req-d910653b-2432-4848-a5b1-55b1a74025e6 req-e358838f-2a49-461f-8c1b-338321c5d2a7 service nova] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Refreshing instance network info cache due to event network-changed-ec1c8d01-1c15-4a89-9205-23f04ec6fc58. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d910653b-2432-4848-a5b1-55b1a74025e6 req-e358838f-2a49-461f-8c1b-338321c5d2a7 service nova] Acquiring lock "refresh_cache-117ea473-f8e3-43c7-8406-d4590422cc0b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.network.neutron [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-75304fcb-33a8-4519-930e-42178f6db253 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Acquiring lock "bf03ba16-e423-4042-a8f8-3ca2d0055617" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-75304fcb-33a8-4519-930e-42178f6db253 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "bf03ba16-e423-4042-a8f8-3ca2d0055617" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-75304fcb-33a8-4519-930e-42178f6db253 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Acquiring lock "bf03ba16-e423-4042-a8f8-3ca2d0055617-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-75304fcb-33a8-4519-930e-42178f6db253 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "bf03ba16-e423-4042-a8f8-3ca2d0055617-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-75304fcb-33a8-4519-930e-42178f6db253 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "bf03ba16-e423-4042-a8f8-3ca2d0055617-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:41 user nova-compute[72081]: INFO nova.compute.manager [None req-75304fcb-33a8-4519-930e-42178f6db253 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Terminating instance
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.compute.manager [None req-75304fcb-33a8-4519-930e-42178f6db253 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.network.neutron [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Updating instance_info_cache with network_info: [{"id": "ec1c8d01-1c15-4a89-9205-23f04ec6fc58", "address": "fa:16:3e:3a:99:8b", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapec1c8d01-1c", "ovs_interfaceid": "ec1c8d01-1c15-4a89-9205-23f04ec6fc58", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Releasing lock "refresh_cache-117ea473-f8e3-43c7-8406-d4590422cc0b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Instance network_info: |[{"id": "ec1c8d01-1c15-4a89-9205-23f04ec6fc58", "address": "fa:16:3e:3a:99:8b", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapec1c8d01-1c", "ovs_interfaceid": "ec1c8d01-1c15-4a89-9205-23f04ec6fc58", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d910653b-2432-4848-a5b1-55b1a74025e6 req-e358838f-2a49-461f-8c1b-338321c5d2a7 service nova] Acquired lock "refresh_cache-117ea473-f8e3-43c7-8406-d4590422cc0b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.network.neutron [req-d910653b-2432-4848-a5b1-55b1a74025e6 req-e358838f-2a49-461f-8c1b-338321c5d2a7 service nova] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Refreshing network info cache for port ec1c8d01-1c15-4a89-9205-23f04ec6fc58 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Start _get_guest_xml network_info=[{"id": "ec1c8d01-1c15-4a89-9205-23f04ec6fc58", "address": "fa:16:3e:3a:99:8b", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapec1c8d01-1c", "ovs_interfaceid": "ec1c8d01-1c15-4a89-9205-23f04ec6fc58", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:40:41 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:40:41 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:40:38Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-1563626038',display_name='tempest-AttachVolumeNegativeTest-server-1563626038',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-1563626038',id=28,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFyKNKT4wgJznr+Ex2kNQaN9dpRqnWsyzDU9MRy0fmokHPPmcvE5+ezeiXGgA2zAoziwbaT10GVLc8cJoTbEoKhsJqM+5rbndvelFILpQi0tIFZnPn0XCK6f6SU5ZH8k7g==',key_name='tempest-keypair-819201420',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='d13546149a4e44e1b782b99e9eb452e8',ramdisk_id='',reservation_id='r-nw8w4ad9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-494105310',owner_user_name='tempest-AttachVolumeNegativeTest-494105310-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:40:40Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d52637cca2114e5e99e688ee371c835a',uuid=117ea473-f8e3-43c7-8406-d4590422cc0b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "ec1c8d01-1c15-4a89-9205-23f04ec6fc58", "address": "fa:16:3e:3a:99:8b", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapec1c8d01-1c", "ovs_interfaceid": "ec1c8d01-1c15-4a89-9205-23f04ec6fc58", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Converting VIF {"id": "ec1c8d01-1c15-4a89-9205-23f04ec6fc58", "address": "fa:16:3e:3a:99:8b", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapec1c8d01-1c", "ovs_interfaceid": "ec1c8d01-1c15-4a89-9205-23f04ec6fc58", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:3a:99:8b,bridge_name='br-int',has_traffic_filtering=True,id=ec1c8d01-1c15-4a89-9205-23f04ec6fc58,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapec1c8d01-1c') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.objects.instance [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lazy-loading 'pci_devices' on Instance uuid 117ea473-f8e3-43c7-8406-d4590422cc0b {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:40:41 user nova-compute[72081]:   <uuid>117ea473-f8e3-43c7-8406-d4590422cc0b</uuid>
Apr 12 03:40:41 user nova-compute[72081]:   <name>instance-0000001c</name>
Apr 12 03:40:41 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:40:41 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:40:41 user nova-compute[72081]:   <metadata>
Apr 12 03:40:41 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:40:41 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:40:41 user nova-compute[72081]:       <nova:name>tempest-AttachVolumeNegativeTest-server-1563626038</nova:name>
Apr 12 03:40:41 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:40:41</nova:creationTime>
Apr 12 03:40:41 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:40:41 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:40:41 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:40:41 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:40:41 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:40:41 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:40:41 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:40:41 user nova-compute[72081]:       <nova:owner>
Apr 12 03:40:41 user nova-compute[72081]:         <nova:user uuid="d52637cca2114e5e99e688ee371c835a">tempest-AttachVolumeNegativeTest-494105310-project-member</nova:user>
Apr 12 03:40:41 user nova-compute[72081]:         <nova:project uuid="d13546149a4e44e1b782b99e9eb452e8">tempest-AttachVolumeNegativeTest-494105310</nova:project>
Apr 12 03:40:41 user nova-compute[72081]:       </nova:owner>
Apr 12 03:40:41 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:40:41 user nova-compute[72081]:       <nova:ports>
Apr 12 03:40:41 user nova-compute[72081]:         <nova:port uuid="ec1c8d01-1c15-4a89-9205-23f04ec6fc58">
Apr 12 03:40:41 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Apr 12 03:40:41 user nova-compute[72081]:         </nova:port>
Apr 12 03:40:41 user nova-compute[72081]:       </nova:ports>
Apr 12 03:40:41 user nova-compute[72081]:     </nova:instance>
Apr 12 03:40:41 user nova-compute[72081]:   </metadata>
Apr 12 03:40:41 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:40:41 user nova-compute[72081]:     <system>
Apr 12 03:40:41 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:40:41 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:40:41 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:40:41 user nova-compute[72081]:       <entry name="serial">117ea473-f8e3-43c7-8406-d4590422cc0b</entry>
Apr 12 03:40:41 user nova-compute[72081]:       <entry name="uuid">117ea473-f8e3-43c7-8406-d4590422cc0b</entry>
Apr 12 03:40:41 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:40:41 user nova-compute[72081]:     </system>
Apr 12 03:40:41 user nova-compute[72081]:   </sysinfo>
Apr 12 03:40:41 user nova-compute[72081]:   <os>
Apr 12 03:40:41 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:40:41 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:40:41 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:40:41 user nova-compute[72081]:   </os>
Apr 12 03:40:41 user nova-compute[72081]:   <features>
Apr 12 03:40:41 user nova-compute[72081]:     <acpi/>
Apr 12 03:40:41 user nova-compute[72081]:     <apic/>
Apr 12 03:40:41 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:40:41 user nova-compute[72081]:   </features>
Apr 12 03:40:41 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:40:41 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:40:41 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:40:41 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:40:41 user nova-compute[72081]:   </clock>
Apr 12 03:40:41 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:40:41 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:40:41 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:40:41 user nova-compute[72081]:   </cpu>
Apr 12 03:40:41 user nova-compute[72081]:   <devices>
Apr 12 03:40:41 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:40:41 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:40:41 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/117ea473-f8e3-43c7-8406-d4590422cc0b/disk"/>
Apr 12 03:40:41 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:40:41 user nova-compute[72081]:     </disk>
Apr 12 03:40:41 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:40:41 user nova-compute[72081]:       <mac address="fa:16:3e:3a:99:8b"/>
Apr 12 03:40:41 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:40:41 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:40:41 user nova-compute[72081]:       <target dev="tapec1c8d01-1c"/>
Apr 12 03:40:41 user nova-compute[72081]:     </interface>
Apr 12 03:40:41 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:40:41 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/117ea473-f8e3-43c7-8406-d4590422cc0b/console.log" append="off"/>
Apr 12 03:40:41 user nova-compute[72081]:     </serial>
Apr 12 03:40:41 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:40:41 user nova-compute[72081]:     <video>
Apr 12 03:40:41 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:40:41 user nova-compute[72081]:     </video>
Apr 12 03:40:41 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:40:41 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:40:41 user nova-compute[72081]:     </rng>
Apr 12 03:40:41 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:40:41 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:40:41 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:40:41 user nova-compute[72081]:     </memballoon>
Apr 12 03:40:41 user nova-compute[72081]:   </devices>
Apr 12 03:40:41 user nova-compute[72081]: </domain>
Apr 12 03:40:41 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:40:38Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-1563626038',display_name='tempest-AttachVolumeNegativeTest-server-1563626038',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-1563626038',id=28,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFyKNKT4wgJznr+Ex2kNQaN9dpRqnWsyzDU9MRy0fmokHPPmcvE5+ezeiXGgA2zAoziwbaT10GVLc8cJoTbEoKhsJqM+5rbndvelFILpQi0tIFZnPn0XCK6f6SU5ZH8k7g==',key_name='tempest-keypair-819201420',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='d13546149a4e44e1b782b99e9eb452e8',ramdisk_id='',reservation_id='r-nw8w4ad9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-494105310',owner_user_name='tempest-AttachVolumeNegativeTest-494105310-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:40:40Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d52637cca2114e5e99e688ee371c835a',uuid=117ea473-f8e3-43c7-8406-d4590422cc0b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "ec1c8d01-1c15-4a89-9205-23f04ec6fc58", "address": "fa:16:3e:3a:99:8b", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapec1c8d01-1c", "ovs_interfaceid": "ec1c8d01-1c15-4a89-9205-23f04ec6fc58", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Converting VIF {"id": "ec1c8d01-1c15-4a89-9205-23f04ec6fc58", "address": "fa:16:3e:3a:99:8b", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapec1c8d01-1c", "ovs_interfaceid": "ec1c8d01-1c15-4a89-9205-23f04ec6fc58", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:3a:99:8b,bridge_name='br-int',has_traffic_filtering=True,id=ec1c8d01-1c15-4a89-9205-23f04ec6fc58,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapec1c8d01-1c') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG os_vif [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:3a:99:8b,bridge_name='br-int',has_traffic_filtering=True,id=ec1c8d01-1c15-4a89-9205-23f04ec6fc58,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapec1c8d01-1c') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapec1c8d01-1c, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapec1c8d01-1c, col_values=(('external_ids', {'iface-id': 'ec1c8d01-1c15-4a89-9205-23f04ec6fc58', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:3a:99:8b', 'vm-uuid': '117ea473-f8e3-43c7-8406-d4590422cc0b'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:41 user nova-compute[72081]: INFO os_vif [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:3a:99:8b,bridge_name='br-int',has_traffic_filtering=True,id=ec1c8d01-1c15-4a89-9205-23f04ec6fc58,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapec1c8d01-1c')
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] No VIF found with MAC fa:16:3e:3a:99:8b, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.compute.manager [req-3f965ce2-73e9-4011-b2c9-f2e6b699b8ab req-c504e01c-462d-4e67-a7e0-97c66fdab005 service nova] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Received event network-vif-unplugged-f258f181-9892-48fa-b054-aff7b58bd433 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3f965ce2-73e9-4011-b2c9-f2e6b699b8ab req-c504e01c-462d-4e67-a7e0-97c66fdab005 service nova] Acquiring lock "bf03ba16-e423-4042-a8f8-3ca2d0055617-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3f965ce2-73e9-4011-b2c9-f2e6b699b8ab req-c504e01c-462d-4e67-a7e0-97c66fdab005 service nova] Lock "bf03ba16-e423-4042-a8f8-3ca2d0055617-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3f965ce2-73e9-4011-b2c9-f2e6b699b8ab req-c504e01c-462d-4e67-a7e0-97c66fdab005 service nova] Lock "bf03ba16-e423-4042-a8f8-3ca2d0055617-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.compute.manager [req-3f965ce2-73e9-4011-b2c9-f2e6b699b8ab req-c504e01c-462d-4e67-a7e0-97c66fdab005 service nova] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] No waiting events found dispatching network-vif-unplugged-f258f181-9892-48fa-b054-aff7b58bd433 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:40:41 user nova-compute[72081]: DEBUG nova.compute.manager [req-3f965ce2-73e9-4011-b2c9-f2e6b699b8ab req-c504e01c-462d-4e67-a7e0-97c66fdab005 service nova] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Received event network-vif-unplugged-f258f181-9892-48fa-b054-aff7b58bd433 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:40:42 user nova-compute[72081]: DEBUG nova.network.neutron [req-d910653b-2432-4848-a5b1-55b1a74025e6 req-e358838f-2a49-461f-8c1b-338321c5d2a7 service nova] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Updated VIF entry in instance network info cache for port ec1c8d01-1c15-4a89-9205-23f04ec6fc58. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:40:42 user nova-compute[72081]: DEBUG nova.network.neutron [req-d910653b-2432-4848-a5b1-55b1a74025e6 req-e358838f-2a49-461f-8c1b-338321c5d2a7 service nova] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Updating instance_info_cache with network_info: [{"id": "ec1c8d01-1c15-4a89-9205-23f04ec6fc58", "address": "fa:16:3e:3a:99:8b", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapec1c8d01-1c", "ovs_interfaceid": "ec1c8d01-1c15-4a89-9205-23f04ec6fc58", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:40:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d910653b-2432-4848-a5b1-55b1a74025e6 req-e358838f-2a49-461f-8c1b-338321c5d2a7 service nova] Releasing lock "refresh_cache-117ea473-f8e3-43c7-8406-d4590422cc0b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:40:42 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Instance destroyed successfully.
Apr 12 03:40:42 user nova-compute[72081]: DEBUG nova.objects.instance [None req-75304fcb-33a8-4519-930e-42178f6db253 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lazy-loading 'resources' on Instance uuid bf03ba16-e423-4042-a8f8-3ca2d0055617 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:40:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-75304fcb-33a8-4519-930e-42178f6db253 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:38:50Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-1208948809',display_name='tempest-VolumesActionsTest-instance-1208948809',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-1208948809',id=25,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:38:57Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='f4f984af8c7048c7ba68c1530be9cc8d',ramdisk_id='',reservation_id='r-f0z4bivk',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesActionsTest-629816748',owner_user_name='tempest-VolumesActionsTest-629816748-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:38:57Z,user_data=None,user_id='45fe6fed1a5a43e5bd0925b92486ec68',uuid=bf03ba16-e423-4042-a8f8-3ca2d0055617,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "f258f181-9892-48fa-b054-aff7b58bd433", "address": "fa:16:3e:ab:86:b7", "network": {"id": "3ec59075-3409-4372-a0ca-6283a5d56ca0", "bridge": "br-int", "label": "tempest-VolumesActionsTest-959291386-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f4f984af8c7048c7ba68c1530be9cc8d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf258f181-98", "ovs_interfaceid": "f258f181-9892-48fa-b054-aff7b58bd433", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:40:42 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-75304fcb-33a8-4519-930e-42178f6db253 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Converting VIF {"id": "f258f181-9892-48fa-b054-aff7b58bd433", "address": "fa:16:3e:ab:86:b7", "network": {"id": "3ec59075-3409-4372-a0ca-6283a5d56ca0", "bridge": "br-int", "label": "tempest-VolumesActionsTest-959291386-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f4f984af8c7048c7ba68c1530be9cc8d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf258f181-98", "ovs_interfaceid": "f258f181-9892-48fa-b054-aff7b58bd433", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:40:42 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-75304fcb-33a8-4519-930e-42178f6db253 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ab:86:b7,bridge_name='br-int',has_traffic_filtering=True,id=f258f181-9892-48fa-b054-aff7b58bd433,network=Network(3ec59075-3409-4372-a0ca-6283a5d56ca0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf258f181-98') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:40:42 user nova-compute[72081]: DEBUG os_vif [None req-75304fcb-33a8-4519-930e-42178f6db253 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:ab:86:b7,bridge_name='br-int',has_traffic_filtering=True,id=f258f181-9892-48fa-b054-aff7b58bd433,network=Network(3ec59075-3409-4372-a0ca-6283a5d56ca0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf258f181-98') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:40:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapf258f181-98, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:40:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:40:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:42 user nova-compute[72081]: INFO os_vif [None req-75304fcb-33a8-4519-930e-42178f6db253 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:ab:86:b7,bridge_name='br-int',has_traffic_filtering=True,id=f258f181-9892-48fa-b054-aff7b58bd433,network=Network(3ec59075-3409-4372-a0ca-6283a5d56ca0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf258f181-98')
Apr 12 03:40:42 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-75304fcb-33a8-4519-930e-42178f6db253 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Deleting instance files /opt/stack/data/nova/instances/bf03ba16-e423-4042-a8f8-3ca2d0055617_del
Apr 12 03:40:42 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-75304fcb-33a8-4519-930e-42178f6db253 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Deletion of /opt/stack/data/nova/instances/bf03ba16-e423-4042-a8f8-3ca2d0055617_del complete
Apr 12 03:40:42 user nova-compute[72081]: INFO nova.compute.manager [None req-75304fcb-33a8-4519-930e-42178f6db253 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Took 0.99 seconds to destroy the instance on the hypervisor.
Apr 12 03:40:42 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-75304fcb-33a8-4519-930e-42178f6db253 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:40:42 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:40:42 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:40:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:43 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:40:43 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Took 0.58 seconds to deallocate network for instance.
Apr 12 03:40:43 user nova-compute[72081]: DEBUG nova.compute.manager [req-7b846477-cd85-4cce-8c4c-734b13b07e15 req-8298a23a-f949-4d95-9eed-b0d4cc6654c9 service nova] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Received event network-vif-deleted-f258f181-9892-48fa-b054-aff7b58bd433 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:40:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-75304fcb-33a8-4519-930e-42178f6db253 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-75304fcb-33a8-4519-930e-42178f6db253 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:43 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-75304fcb-33a8-4519-930e-42178f6db253 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:40:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:43 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-75304fcb-33a8-4519-930e-42178f6db253 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:40:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-75304fcb-33a8-4519-930e-42178f6db253 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.499s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:43 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-75304fcb-33a8-4519-930e-42178f6db253 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Deleted allocations for instance bf03ba16-e423-4042-a8f8-3ca2d0055617
Apr 12 03:40:43 user nova-compute[72081]: DEBUG nova.compute.manager [req-61a8b8a1-c51b-4a8a-835b-a4ff8ce4ae89 req-89ecf3b0-a0a4-4020-8b7e-09689e686869 service nova] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Received event network-vif-plugged-f258f181-9892-48fa-b054-aff7b58bd433 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:40:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-61a8b8a1-c51b-4a8a-835b-a4ff8ce4ae89 req-89ecf3b0-a0a4-4020-8b7e-09689e686869 service nova] Acquiring lock "bf03ba16-e423-4042-a8f8-3ca2d0055617-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-61a8b8a1-c51b-4a8a-835b-a4ff8ce4ae89 req-89ecf3b0-a0a4-4020-8b7e-09689e686869 service nova] Lock "bf03ba16-e423-4042-a8f8-3ca2d0055617-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-61a8b8a1-c51b-4a8a-835b-a4ff8ce4ae89 req-89ecf3b0-a0a4-4020-8b7e-09689e686869 service nova] Lock "bf03ba16-e423-4042-a8f8-3ca2d0055617-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:43 user nova-compute[72081]: DEBUG nova.compute.manager [req-61a8b8a1-c51b-4a8a-835b-a4ff8ce4ae89 req-89ecf3b0-a0a4-4020-8b7e-09689e686869 service nova] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] No waiting events found dispatching network-vif-plugged-f258f181-9892-48fa-b054-aff7b58bd433 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:40:43 user nova-compute[72081]: WARNING nova.compute.manager [req-61a8b8a1-c51b-4a8a-835b-a4ff8ce4ae89 req-89ecf3b0-a0a4-4020-8b7e-09689e686869 service nova] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Received unexpected event network-vif-plugged-f258f181-9892-48fa-b054-aff7b58bd433 for instance with vm_state deleted and task_state None.
Apr 12 03:40:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-75304fcb-33a8-4519-930e-42178f6db253 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "bf03ba16-e423-4042-a8f8-3ca2d0055617" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.381s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:44 user nova-compute[72081]: DEBUG nova.compute.manager [req-e8866a42-66b5-4f6e-978b-8d321e32ee96 req-8ce3084f-94b0-4426-8508-0d62c32d5e78 service nova] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Received event network-vif-plugged-ec1c8d01-1c15-4a89-9205-23f04ec6fc58 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:40:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e8866a42-66b5-4f6e-978b-8d321e32ee96 req-8ce3084f-94b0-4426-8508-0d62c32d5e78 service nova] Acquiring lock "117ea473-f8e3-43c7-8406-d4590422cc0b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e8866a42-66b5-4f6e-978b-8d321e32ee96 req-8ce3084f-94b0-4426-8508-0d62c32d5e78 service nova] Lock "117ea473-f8e3-43c7-8406-d4590422cc0b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e8866a42-66b5-4f6e-978b-8d321e32ee96 req-8ce3084f-94b0-4426-8508-0d62c32d5e78 service nova] Lock "117ea473-f8e3-43c7-8406-d4590422cc0b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:44 user nova-compute[72081]: DEBUG nova.compute.manager [req-e8866a42-66b5-4f6e-978b-8d321e32ee96 req-8ce3084f-94b0-4426-8508-0d62c32d5e78 service nova] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] No waiting events found dispatching network-vif-plugged-ec1c8d01-1c15-4a89-9205-23f04ec6fc58 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:40:44 user nova-compute[72081]: WARNING nova.compute.manager [req-e8866a42-66b5-4f6e-978b-8d321e32ee96 req-8ce3084f-94b0-4426-8508-0d62c32d5e78 service nova] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Received unexpected event network-vif-plugged-ec1c8d01-1c15-4a89-9205-23f04ec6fc58 for instance with vm_state building and task_state spawning.
Apr 12 03:40:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:45 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270845.9238513, 117ea473-f8e3-43c7-8406-d4590422cc0b => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:40:45 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] VM Resumed (Lifecycle Event)
Apr 12 03:40:45 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:40:45 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:40:45 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Instance spawned successfully.
Apr 12 03:40:45 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:40:45 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:40:45 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:40:45 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:40:45 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:40:45 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:40:45 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:40:45 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:40:45 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:40:45 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:40:45 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270845.924919, 117ea473-f8e3-43c7-8406-d4590422cc0b => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:40:45 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] VM Started (Lifecycle Event)
Apr 12 03:40:46 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:40:46 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:40:46 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:40:46 user nova-compute[72081]: INFO nova.compute.manager [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Took 6.49 seconds to spawn the instance on the hypervisor.
Apr 12 03:40:46 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:40:46 user nova-compute[72081]: DEBUG nova.compute.manager [req-abf830d9-163a-481a-b461-99a23151cd72 req-5feadf49-67af-4965-86e5-759ba1a09c28 service nova] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Received event network-vif-plugged-ec1c8d01-1c15-4a89-9205-23f04ec6fc58 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:40:46 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-abf830d9-163a-481a-b461-99a23151cd72 req-5feadf49-67af-4965-86e5-759ba1a09c28 service nova] Acquiring lock "117ea473-f8e3-43c7-8406-d4590422cc0b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:46 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-abf830d9-163a-481a-b461-99a23151cd72 req-5feadf49-67af-4965-86e5-759ba1a09c28 service nova] Lock "117ea473-f8e3-43c7-8406-d4590422cc0b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:46 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-abf830d9-163a-481a-b461-99a23151cd72 req-5feadf49-67af-4965-86e5-759ba1a09c28 service nova] Lock "117ea473-f8e3-43c7-8406-d4590422cc0b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:46 user nova-compute[72081]: DEBUG nova.compute.manager [req-abf830d9-163a-481a-b461-99a23151cd72 req-5feadf49-67af-4965-86e5-759ba1a09c28 service nova] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] No waiting events found dispatching network-vif-plugged-ec1c8d01-1c15-4a89-9205-23f04ec6fc58 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:40:46 user nova-compute[72081]: WARNING nova.compute.manager [req-abf830d9-163a-481a-b461-99a23151cd72 req-5feadf49-67af-4965-86e5-759ba1a09c28 service nova] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Received unexpected event network-vif-plugged-ec1c8d01-1c15-4a89-9205-23f04ec6fc58 for instance with vm_state building and task_state spawning.
Apr 12 03:40:46 user nova-compute[72081]: INFO nova.compute.manager [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Took 7.35 seconds to build instance.
Apr 12 03:40:46 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f0683b23-5834-4ba3-8f20-34a5cfca0c5d tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "117ea473-f8e3-43c7-8406-d4590422cc0b" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.445s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6641096e-8d16-449e-a5c3-9ec019acfb82 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquiring lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6641096e-8d16-449e-a5c3-9ec019acfb82 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6641096e-8d16-449e-a5c3-9ec019acfb82 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquiring lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6641096e-8d16-449e-a5c3-9ec019acfb82 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6641096e-8d16-449e-a5c3-9ec019acfb82 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:52 user nova-compute[72081]: INFO nova.compute.manager [None req-6641096e-8d16-449e-a5c3-9ec019acfb82 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Terminating instance
Apr 12 03:40:52 user nova-compute[72081]: DEBUG nova.compute.manager [None req-6641096e-8d16-449e-a5c3-9ec019acfb82 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:40:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:52 user nova-compute[72081]: DEBUG nova.compute.manager [req-888204f5-4d82-4d5c-a0ba-32dc29db4824 req-65709e61-9b6d-45ca-ae3e-44007e6ce170 service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Received event network-vif-unplugged-892ff129-c9ca-4295-be61-993b81b855fb {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:40:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-888204f5-4d82-4d5c-a0ba-32dc29db4824 req-65709e61-9b6d-45ca-ae3e-44007e6ce170 service nova] Acquiring lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-888204f5-4d82-4d5c-a0ba-32dc29db4824 req-65709e61-9b6d-45ca-ae3e-44007e6ce170 service nova] Lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-888204f5-4d82-4d5c-a0ba-32dc29db4824 req-65709e61-9b6d-45ca-ae3e-44007e6ce170 service nova] Lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:52 user nova-compute[72081]: DEBUG nova.compute.manager [req-888204f5-4d82-4d5c-a0ba-32dc29db4824 req-65709e61-9b6d-45ca-ae3e-44007e6ce170 service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] No waiting events found dispatching network-vif-unplugged-892ff129-c9ca-4295-be61-993b81b855fb {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:40:52 user nova-compute[72081]: DEBUG nova.compute.manager [req-888204f5-4d82-4d5c-a0ba-32dc29db4824 req-65709e61-9b6d-45ca-ae3e-44007e6ce170 service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Received event network-vif-unplugged-892ff129-c9ca-4295-be61-993b81b855fb for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:40:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:53 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Instance destroyed successfully.
Apr 12 03:40:53 user nova-compute[72081]: DEBUG nova.objects.instance [None req-6641096e-8d16-449e-a5c3-9ec019acfb82 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lazy-loading 'resources' on Instance uuid a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:40:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-6641096e-8d16-449e-a5c3-9ec019acfb82 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:35:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-2065419898',display_name='tempest-VolumesAdminNegativeTest-server-2065419898',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-2065419898',id=19,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEZmg9av5nHzz+WM/CeO9E4jVqkDumG/6+9A6cHxDvsmxb0GgBIDFr9i3AR8FMpBKXAtQyO3hr0qIi3g2HX/TjWxD4YibE1LO5tXDCbvlMnQhnENYSPZrfx3AZVOxrgL2g==',key_name='tempest-keypair-315148094',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:35:22Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='f4cce206e2b94b03ade2d310373c5ca2',ramdisk_id='',reservation_id='r-i8pcads2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesAdminNegativeTest-2105871059',owner_user_name='tempest-VolumesAdminNegativeTest-2105871059-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:35:23Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e2f496e5fe694d46a9803f797743a7dc',uuid=a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "892ff129-c9ca-4295-be61-993b81b855fb", "address": "fa:16:3e:f0:b1:a0", "network": {"id": "263f9508-8639-4289-9210-f9807acf6766", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-337713645-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.195", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f4cce206e2b94b03ade2d310373c5ca2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap892ff129-c9", "ovs_interfaceid": "892ff129-c9ca-4295-be61-993b81b855fb", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:40:53 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-6641096e-8d16-449e-a5c3-9ec019acfb82 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Converting VIF {"id": "892ff129-c9ca-4295-be61-993b81b855fb", "address": "fa:16:3e:f0:b1:a0", "network": {"id": "263f9508-8639-4289-9210-f9807acf6766", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-337713645-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.195", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f4cce206e2b94b03ade2d310373c5ca2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap892ff129-c9", "ovs_interfaceid": "892ff129-c9ca-4295-be61-993b81b855fb", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:40:53 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-6641096e-8d16-449e-a5c3-9ec019acfb82 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:f0:b1:a0,bridge_name='br-int',has_traffic_filtering=True,id=892ff129-c9ca-4295-be61-993b81b855fb,network=Network(263f9508-8639-4289-9210-f9807acf6766),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap892ff129-c9') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:40:53 user nova-compute[72081]: DEBUG os_vif [None req-6641096e-8d16-449e-a5c3-9ec019acfb82 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:f0:b1:a0,bridge_name='br-int',has_traffic_filtering=True,id=892ff129-c9ca-4295-be61-993b81b855fb,network=Network(263f9508-8639-4289-9210-f9807acf6766),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap892ff129-c9') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:40:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap892ff129-c9, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:40:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:40:53 user nova-compute[72081]: INFO os_vif [None req-6641096e-8d16-449e-a5c3-9ec019acfb82 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:f0:b1:a0,bridge_name='br-int',has_traffic_filtering=True,id=892ff129-c9ca-4295-be61-993b81b855fb,network=Network(263f9508-8639-4289-9210-f9807acf6766),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap892ff129-c9')
Apr 12 03:40:53 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-6641096e-8d16-449e-a5c3-9ec019acfb82 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Deleting instance files /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6_del
Apr 12 03:40:53 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-6641096e-8d16-449e-a5c3-9ec019acfb82 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Deletion of /opt/stack/data/nova/instances/a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6_del complete
Apr 12 03:40:53 user nova-compute[72081]: INFO nova.compute.manager [None req-6641096e-8d16-449e-a5c3-9ec019acfb82 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Took 0.91 seconds to destroy the instance on the hypervisor.
Apr 12 03:40:53 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-6641096e-8d16-449e-a5c3-9ec019acfb82 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:40:53 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:40:53 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:40:54 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:40:54 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Took 0.77 seconds to deallocate network for instance.
Apr 12 03:40:54 user nova-compute[72081]: DEBUG nova.compute.manager [req-83e43945-facb-466d-9cde-a81b590669ff req-1567a42f-37fd-4c2c-9763-ff4497b78f8d service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Received event network-vif-deleted-892ff129-c9ca-4295-be61-993b81b855fb {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:40:54 user nova-compute[72081]: INFO nova.compute.manager [req-83e43945-facb-466d-9cde-a81b590669ff req-1567a42f-37fd-4c2c-9763-ff4497b78f8d service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Neutron deleted interface 892ff129-c9ca-4295-be61-993b81b855fb; detaching it from the instance and deleting it from the info cache
Apr 12 03:40:54 user nova-compute[72081]: DEBUG nova.network.neutron [req-83e43945-facb-466d-9cde-a81b590669ff req-1567a42f-37fd-4c2c-9763-ff4497b78f8d service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:40:54 user nova-compute[72081]: DEBUG nova.compute.manager [req-83e43945-facb-466d-9cde-a81b590669ff req-1567a42f-37fd-4c2c-9763-ff4497b78f8d service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Detach interface failed, port_id=892ff129-c9ca-4295-be61-993b81b855fb, reason: Instance a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6 could not be found. {{(pid=72081) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 12 03:40:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6641096e-8d16-449e-a5c3-9ec019acfb82 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6641096e-8d16-449e-a5c3-9ec019acfb82 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:54 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-6641096e-8d16-449e-a5c3-9ec019acfb82 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:40:54 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-6641096e-8d16-449e-a5c3-9ec019acfb82 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:40:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6641096e-8d16-449e-a5c3-9ec019acfb82 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.344s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:54 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-6641096e-8d16-449e-a5c3-9ec019acfb82 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Deleted allocations for instance a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6
Apr 12 03:40:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6641096e-8d16-449e-a5c3-9ec019acfb82 tempest-VolumesAdminNegativeTest-2105871059 tempest-VolumesAdminNegativeTest-2105871059-project-member] Lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.223s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:54 user nova-compute[72081]: DEBUG nova.compute.manager [req-646c36a6-960f-4adc-b189-49f6f99b0144 req-91826a65-81b8-4d6b-8f07-5ebbfb968e6a service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Received event network-vif-plugged-892ff129-c9ca-4295-be61-993b81b855fb {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:40:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-646c36a6-960f-4adc-b189-49f6f99b0144 req-91826a65-81b8-4d6b-8f07-5ebbfb968e6a service nova] Acquiring lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:40:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-646c36a6-960f-4adc-b189-49f6f99b0144 req-91826a65-81b8-4d6b-8f07-5ebbfb968e6a service nova] Lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:40:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-646c36a6-960f-4adc-b189-49f6f99b0144 req-91826a65-81b8-4d6b-8f07-5ebbfb968e6a service nova] Lock "a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:40:54 user nova-compute[72081]: DEBUG nova.compute.manager [req-646c36a6-960f-4adc-b189-49f6f99b0144 req-91826a65-81b8-4d6b-8f07-5ebbfb968e6a service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] No waiting events found dispatching network-vif-plugged-892ff129-c9ca-4295-be61-993b81b855fb {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:40:54 user nova-compute[72081]: WARNING nova.compute.manager [req-646c36a6-960f-4adc-b189-49f6f99b0144 req-91826a65-81b8-4d6b-8f07-5ebbfb968e6a service nova] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Received unexpected event network-vif-plugged-892ff129-c9ca-4295-be61-993b81b855fb for instance with vm_state deleted and task_state None.
Apr 12 03:40:57 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681270842.478901, bf03ba16-e423-4042-a8f8-3ca2d0055617 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:40:57 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] VM Stopped (Lifecycle Event)
Apr 12 03:40:57 user nova-compute[72081]: DEBUG nova.compute.manager [None req-82962acf-18dc-40f5-9dc3-dcd769e54182 None None] [instance: bf03ba16-e423-4042-a8f8-3ca2d0055617] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:40:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:40:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:00 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:41:00 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Cleaning up deleted instances with incomplete migration  {{(pid=72081) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 12 03:41:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:04 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:41:04 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:41:04 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:41:05 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:41:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:05 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:41:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/117ea473-f8e3-43c7-8406-d4590422cc0b/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/117ea473-f8e3-43c7-8406-d4590422cc0b/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/117ea473-f8e3-43c7-8406-d4590422cc0b/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/117ea473-f8e3-43c7-8406-d4590422cc0b/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:41:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdd (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:41:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:41:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:41:08 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:41:08 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:41:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=7808MB free_disk=26.010208129882812GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:41:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:08 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681270853.3488343, a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:41:08 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] VM Stopped (Lifecycle Event)
Apr 12 03:41:08 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1c166b57-dd6c-49c4-82c0-81055f73d0d6 None None] [instance: a7f8feb8-9f2d-4cf4-b973-60b04c5d02b6] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:41:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 2d0904c3-74e0-47bb-a832-758a7035b2f5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:41:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance bcd267d2-5b2c-4ed2-b634-0210fae94dfd actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:41:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 48977eb2-ce43-4577-a1cb-340e35703dbd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:41:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 1eef2c91-dfad-43aa-b193-25970db83b79 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:41:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance f73fa883-75f0-4593-a89c-d009c6d4ebf0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:41:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 3d4407f4-e078-4777-a110-cb3f922be2e8 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:41:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance d211dca2-af04-4dd3-b747-cc3bbd8d310b actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:41:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:41:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 117ea473-f8e3-43c7-8406-d4590422cc0b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:41:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:41:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=1664MB phys_disk=40GB used_disk=6GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:41:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:08 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Refreshing inventories for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 12 03:41:08 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Updating ProviderTree inventory for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 12 03:41:08 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Updating inventory in ProviderTree for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 12 03:41:08 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Refreshing aggregate associations for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6, aggregates: None {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 12 03:41:08 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Refreshing trait associations for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6, traits: COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_USB,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_RESCUE_BFV,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE41,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_STORAGE_BUS_FDC,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_STORAGE_BUS_IDE,COMPUTE_TRUSTED_CERTS,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_GRAPHICS_MODEL_CIRRUS,HW_CPU_X86_SSSE3,HW_CPU_X86_SSE,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_EXTEND,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_STORAGE_BUS_SATA,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_GRAPHICS_MODEL_QXL,HW_CPU_X86_SSE42,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_ACCELERATORS,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,HW_CPU_X86_MMX,COMPUTE_NODE,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_DEVICE_TAGGING,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_VIOMMU_MODEL_AUTO {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 12 03:41:08 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:41:08 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:41:08 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:41:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.748s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:10 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:41:10 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:41:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:41:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Rebuilding the list of instances to heal {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 12 03:41:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-2d0904c3-74e0-47bb-a832-758a7035b2f5" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:41:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-2d0904c3-74e0-47bb-a832-758a7035b2f5" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:41:10 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 03:41:10 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lazy-loading 'info_cache' on Instance uuid 2d0904c3-74e0-47bb-a832-758a7035b2f5 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:41:11 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Updating instance_info_cache with network_info: [{"id": "dce84390-a2f8-4047-b513-eec5845a2399", "address": "fa:16:3e:6e:2e:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdce84390-a2", "ovs_interfaceid": "dce84390-a2f8-4047-b513-eec5845a2399", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:41:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-2d0904c3-74e0-47bb-a832-758a7035b2f5" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:41:11 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 03:41:11 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:41:11 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:41:11 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:41:11 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:41:11 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:41:11 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:41:11 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Cleaning up deleted instances {{(pid=72081) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 12 03:41:11 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] There are 0 instances to clean {{(pid=72081) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 12 03:41:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "06481254-722a-4053-a6af-11f5dcc68636" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "06481254-722a-4053-a6af-11f5dcc68636" acquired by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:17 user nova-compute[72081]: INFO nova.compute.manager [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Unshelving
Apr 12 03:41:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:18 user nova-compute[72081]: DEBUG nova.objects.instance [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lazy-loading 'pci_requests' on Instance uuid 06481254-722a-4053-a6af-11f5dcc68636 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:41:18 user nova-compute[72081]: DEBUG nova.objects.instance [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lazy-loading 'numa_topology' on Instance uuid 06481254-722a-4053-a6af-11f5dcc68636 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:41:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:41:18 user nova-compute[72081]: INFO nova.compute.claims [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Claim successful on node user
Apr 12 03:41:18 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:41:18 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:41:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.378s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:18 user nova-compute[72081]: INFO nova.network.neutron [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Updating port 5861aafc-4312-4d82-965c-2d1d993d8ea0 with attributes {'binding:host_id': 'user', 'device_owner': 'compute:nova'}
Apr 12 03:41:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "refresh_cache-06481254-722a-4053-a6af-11f5dcc68636" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:41:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquired lock "refresh_cache-06481254-722a-4053-a6af-11f5dcc68636" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:41:19 user nova-compute[72081]: DEBUG nova.network.neutron [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:41:19 user nova-compute[72081]: DEBUG nova.compute.manager [req-b8a7aa5b-a8fc-45f1-90f1-574e7bd16241 req-7a88be42-e5bb-46f8-a669-662a0e0bf324 service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Received event network-changed-5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:41:19 user nova-compute[72081]: DEBUG nova.compute.manager [req-b8a7aa5b-a8fc-45f1-90f1-574e7bd16241 req-7a88be42-e5bb-46f8-a669-662a0e0bf324 service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Refreshing instance network info cache due to event network-changed-5861aafc-4312-4d82-965c-2d1d993d8ea0. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:41:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b8a7aa5b-a8fc-45f1-90f1-574e7bd16241 req-7a88be42-e5bb-46f8-a669-662a0e0bf324 service nova] Acquiring lock "refresh_cache-06481254-722a-4053-a6af-11f5dcc68636" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:41:19 user nova-compute[72081]: DEBUG nova.network.neutron [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Updating instance_info_cache with network_info: [{"id": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "address": "fa:16:3e:c7:7c:eb", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.227", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5861aafc-43", "ovs_interfaceid": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:41:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Releasing lock "refresh_cache-06481254-722a-4053-a6af-11f5dcc68636" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:41:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:41:19 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Creating image(s)
Apr 12 03:41:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "/opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "/opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "/opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:19 user nova-compute[72081]: DEBUG nova.objects.instance [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lazy-loading 'trusted_certs' on Instance uuid 06481254-722a-4053-a6af-11f5dcc68636 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:41:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b8a7aa5b-a8fc-45f1-90f1-574e7bd16241 req-7a88be42-e5bb-46f8-a669-662a0e0bf324 service nova] Acquired lock "refresh_cache-06481254-722a-4053-a6af-11f5dcc68636" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:41:19 user nova-compute[72081]: DEBUG nova.network.neutron [req-b8a7aa5b-a8fc-45f1-90f1-574e7bd16241 req-7a88be42-e5bb-46f8-a669-662a0e0bf324 service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Refreshing network info cache for port 5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:41:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "df2c896d566356f451d4aa39269f45190fe99704" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "df2c896d566356f451d4aa39269f45190fe99704" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/df2c896d566356f451d4aa39269f45190fe99704.part --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/df2c896d566356f451d4aa39269f45190fe99704.part --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:20 user nova-compute[72081]: DEBUG nova.virt.images [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] 291c6f89-241c-4edc-a92d-3497738ac9d3 was qcow2, converting to raw {{(pid=72081) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 12 03:41:20 user nova-compute[72081]: DEBUG nova.privsep.utils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72081) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 12 03:41:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/df2c896d566356f451d4aa39269f45190fe99704.part /opt/stack/data/nova/instances/_base/df2c896d566356f451d4aa39269f45190fe99704.converted {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:21 user nova-compute[72081]: DEBUG nova.network.neutron [req-b8a7aa5b-a8fc-45f1-90f1-574e7bd16241 req-7a88be42-e5bb-46f8-a669-662a0e0bf324 service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Updated VIF entry in instance network info cache for port 5861aafc-4312-4d82-965c-2d1d993d8ea0. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:41:21 user nova-compute[72081]: DEBUG nova.network.neutron [req-b8a7aa5b-a8fc-45f1-90f1-574e7bd16241 req-7a88be42-e5bb-46f8-a669-662a0e0bf324 service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Updating instance_info_cache with network_info: [{"id": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "address": "fa:16:3e:c7:7c:eb", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.227", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5861aafc-43", "ovs_interfaceid": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:41:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b8a7aa5b-a8fc-45f1-90f1-574e7bd16241 req-7a88be42-e5bb-46f8-a669-662a0e0bf324 service nova] Releasing lock "refresh_cache-06481254-722a-4053-a6af-11f5dcc68636" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:41:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/df2c896d566356f451d4aa39269f45190fe99704.part /opt/stack/data/nova/instances/_base/df2c896d566356f451d4aa39269f45190fe99704.converted" returned: 0 in 0.420s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/df2c896d566356f451d4aa39269f45190fe99704.converted --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/df2c896d566356f451d4aa39269f45190fe99704.converted --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "df2c896d566356f451d4aa39269f45190fe99704" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.416s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/df2c896d566356f451d4aa39269f45190fe99704 --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/df2c896d566356f451d4aa39269f45190fe99704 --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "df2c896d566356f451d4aa39269f45190fe99704" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "df2c896d566356f451d4aa39269f45190fe99704" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/df2c896d566356f451d4aa39269f45190fe99704 --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/df2c896d566356f451d4aa39269f45190fe99704 --force-share --output=json" returned: 0 in 0.132s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/df2c896d566356f451d4aa39269f45190fe99704,backing_fmt=raw /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/df2c896d566356f451d4aa39269f45190fe99704,backing_fmt=raw /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk 1073741824" returned: 0 in 0.049s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "df2c896d566356f451d4aa39269f45190fe99704" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.187s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/df2c896d566356f451d4aa39269f45190fe99704 --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/df2c896d566356f451d4aa39269f45190fe99704 --force-share --output=json" returned: 0 in 0.131s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:21 user nova-compute[72081]: DEBUG nova.objects.instance [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lazy-loading 'migration_context' on Instance uuid 06481254-722a-4053-a6af-11f5dcc68636 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:41:21 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Rebasing disk image.
Apr 12 03:41:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.145s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Running cmd (subprocess): qemu-img rebase -b /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df -F raw /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CMD "qemu-img rebase -b /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df -F raw /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk" returned: 0 in 2.451s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Ensure instance console log exists: /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Start _get_guest_xml network_info=[{"id": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "address": "fa:16:3e:c7:7c:eb", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.227", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5861aafc-43", "ovs_interfaceid": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='f6817d77f9c524d841657684701446d9',container_format='bare',created_at=2023-04-12T03:39:25Z,direct_url=<?>,disk_format='qcow2',id=291c6f89-241c-4edc-a92d-3497738ac9d3,min_disk=1,min_ram=0,name='tempest-AttachVolumeShelveTestJSON-server-405065337-shelved',owner='738ebb6b798c4974aa46f9bbe3828279',properties=ImageMetaProps,protected=<?>,size=38600704,status='active',tags=<?>,updated_at=2023-04-12T03:39:29Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:41:24 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:41:24 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='f6817d77f9c524d841657684701446d9',container_format='bare',created_at=2023-04-12T03:39:25Z,direct_url=<?>,disk_format='qcow2',id=291c6f89-241c-4edc-a92d-3497738ac9d3,min_disk=1,min_ram=0,name='tempest-AttachVolumeShelveTestJSON-server-405065337-shelved',owner='738ebb6b798c4974aa46f9bbe3828279',properties=ImageMetaProps,protected=<?>,size=38600704,status='active',tags=<?>,updated_at=2023-04-12T03:39:29Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.objects.instance [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lazy-loading 'vcpu_model' on Instance uuid 06481254-722a-4053-a6af-11f5dcc68636 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-12T03:37:30Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-405065337',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-405065337',id=20,image_ref='291c6f89-241c-4edc-a92d-3497738ac9d3',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name='tempest-keypair-2125155953',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:37:37Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='738ebb6b798c4974aa46f9bbe3828279',ramdisk_id='',reservation_id='r-jbxvhjdl',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-2139975165',owner_user_name='tempest-AttachVolumeShelveTestJSON-2139975165-project-member',shelved_at='2023-04-12T03:39:29.211136',shelved_host='user',shelved_image_id='291c6f89-241c-4edc-a92d-3497738ac9d3'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:41:18Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='de1fe5db01624224ab7b615c908d990b',uuid=06481254-722a-4053-a6af-11f5dcc68636,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "address": "fa:16:3e:c7:7c:eb", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.227", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5861aafc-43", "ovs_interfaceid": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Converting VIF {"id": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "address": "fa:16:3e:c7:7c:eb", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.227", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5861aafc-43", "ovs_interfaceid": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:c7:7c:eb,bridge_name='br-int',has_traffic_filtering=True,id=5861aafc-4312-4d82-965c-2d1d993d8ea0,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5861aafc-43') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.objects.instance [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lazy-loading 'pci_devices' on Instance uuid 06481254-722a-4053-a6af-11f5dcc68636 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:41:24 user nova-compute[72081]:   <uuid>06481254-722a-4053-a6af-11f5dcc68636</uuid>
Apr 12 03:41:24 user nova-compute[72081]:   <name>instance-00000014</name>
Apr 12 03:41:24 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:41:24 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:41:24 user nova-compute[72081]:   <metadata>
Apr 12 03:41:24 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:41:24 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:41:24 user nova-compute[72081]:       <nova:name>tempest-AttachVolumeShelveTestJSON-server-405065337</nova:name>
Apr 12 03:41:24 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:41:24</nova:creationTime>
Apr 12 03:41:24 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:41:24 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:41:24 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:41:24 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:41:24 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:41:24 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:41:24 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:41:24 user nova-compute[72081]:       <nova:owner>
Apr 12 03:41:24 user nova-compute[72081]:         <nova:user uuid="de1fe5db01624224ab7b615c908d990b">tempest-AttachVolumeShelveTestJSON-2139975165-project-member</nova:user>
Apr 12 03:41:24 user nova-compute[72081]:         <nova:project uuid="738ebb6b798c4974aa46f9bbe3828279">tempest-AttachVolumeShelveTestJSON-2139975165</nova:project>
Apr 12 03:41:24 user nova-compute[72081]:       </nova:owner>
Apr 12 03:41:24 user nova-compute[72081]:       <nova:root type="image" uuid="291c6f89-241c-4edc-a92d-3497738ac9d3"/>
Apr 12 03:41:24 user nova-compute[72081]:       <nova:ports>
Apr 12 03:41:24 user nova-compute[72081]:         <nova:port uuid="5861aafc-4312-4d82-965c-2d1d993d8ea0">
Apr 12 03:41:24 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Apr 12 03:41:24 user nova-compute[72081]:         </nova:port>
Apr 12 03:41:24 user nova-compute[72081]:       </nova:ports>
Apr 12 03:41:24 user nova-compute[72081]:     </nova:instance>
Apr 12 03:41:24 user nova-compute[72081]:   </metadata>
Apr 12 03:41:24 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:41:24 user nova-compute[72081]:     <system>
Apr 12 03:41:24 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:41:24 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:41:24 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:41:24 user nova-compute[72081]:       <entry name="serial">06481254-722a-4053-a6af-11f5dcc68636</entry>
Apr 12 03:41:24 user nova-compute[72081]:       <entry name="uuid">06481254-722a-4053-a6af-11f5dcc68636</entry>
Apr 12 03:41:24 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:41:24 user nova-compute[72081]:     </system>
Apr 12 03:41:24 user nova-compute[72081]:   </sysinfo>
Apr 12 03:41:24 user nova-compute[72081]:   <os>
Apr 12 03:41:24 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:41:24 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:41:24 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:41:24 user nova-compute[72081]:   </os>
Apr 12 03:41:24 user nova-compute[72081]:   <features>
Apr 12 03:41:24 user nova-compute[72081]:     <acpi/>
Apr 12 03:41:24 user nova-compute[72081]:     <apic/>
Apr 12 03:41:24 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:41:24 user nova-compute[72081]:   </features>
Apr 12 03:41:24 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:41:24 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:41:24 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:41:24 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:41:24 user nova-compute[72081]:   </clock>
Apr 12 03:41:24 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:41:24 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:41:24 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:41:24 user nova-compute[72081]:   </cpu>
Apr 12 03:41:24 user nova-compute[72081]:   <devices>
Apr 12 03:41:24 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:41:24 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:41:24 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk"/>
Apr 12 03:41:24 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:41:24 user nova-compute[72081]:     </disk>
Apr 12 03:41:24 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:41:24 user nova-compute[72081]:       <mac address="fa:16:3e:c7:7c:eb"/>
Apr 12 03:41:24 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:41:24 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:41:24 user nova-compute[72081]:       <target dev="tap5861aafc-43"/>
Apr 12 03:41:24 user nova-compute[72081]:     </interface>
Apr 12 03:41:24 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:41:24 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/console.log" append="off"/>
Apr 12 03:41:24 user nova-compute[72081]:     </serial>
Apr 12 03:41:24 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:41:24 user nova-compute[72081]:     <video>
Apr 12 03:41:24 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:41:24 user nova-compute[72081]:     </video>
Apr 12 03:41:24 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:41:24 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:41:24 user nova-compute[72081]:     </rng>
Apr 12 03:41:24 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:41:24 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:41:24 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:41:24 user nova-compute[72081]:     </memballoon>
Apr 12 03:41:24 user nova-compute[72081]:   </devices>
Apr 12 03:41:24 user nova-compute[72081]: </domain>
Apr 12 03:41:24 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-12T03:37:30Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-405065337',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-405065337',id=20,image_ref='291c6f89-241c-4edc-a92d-3497738ac9d3',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name='tempest-keypair-2125155953',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:37:37Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='738ebb6b798c4974aa46f9bbe3828279',ramdisk_id='',reservation_id='r-jbxvhjdl',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-2139975165',owner_user_name='tempest-AttachVolumeShelveTestJSON-2139975165-project-member',shelved_at='2023-04-12T03:39:29.211136',shelved_host='user',shelved_image_id='291c6f89-241c-4edc-a92d-3497738ac9d3'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:41:18Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='de1fe5db01624224ab7b615c908d990b',uuid=06481254-722a-4053-a6af-11f5dcc68636,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "address": "fa:16:3e:c7:7c:eb", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.227", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5861aafc-43", "ovs_interfaceid": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Converting VIF {"id": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "address": "fa:16:3e:c7:7c:eb", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.227", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5861aafc-43", "ovs_interfaceid": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:c7:7c:eb,bridge_name='br-int',has_traffic_filtering=True,id=5861aafc-4312-4d82-965c-2d1d993d8ea0,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5861aafc-43') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG os_vif [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:c7:7c:eb,bridge_name='br-int',has_traffic_filtering=True,id=5861aafc-4312-4d82-965c-2d1d993d8ea0,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5861aafc-43') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap5861aafc-43, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap5861aafc-43, col_values=(('external_ids', {'iface-id': '5861aafc-4312-4d82-965c-2d1d993d8ea0', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:c7:7c:eb', 'vm-uuid': '06481254-722a-4053-a6af-11f5dcc68636'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:24 user nova-compute[72081]: INFO os_vif [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:c7:7c:eb,bridge_name='br-int',has_traffic_filtering=True,id=5861aafc-4312-4d82-965c-2d1d993d8ea0,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5861aafc-43')
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:41:24 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] No VIF found with MAC fa:16:3e:c7:7c:eb, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:41:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-1d671a8c-8e05-4d95-b9bc-50d08a636c4f req-e6bd2bfa-338e-4ec3-98ba-ca6c0710eb1a service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Received event network-vif-plugged-5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:41:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1d671a8c-8e05-4d95-b9bc-50d08a636c4f req-e6bd2bfa-338e-4ec3-98ba-ca6c0710eb1a service nova] Acquiring lock "06481254-722a-4053-a6af-11f5dcc68636-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1d671a8c-8e05-4d95-b9bc-50d08a636c4f req-e6bd2bfa-338e-4ec3-98ba-ca6c0710eb1a service nova] Lock "06481254-722a-4053-a6af-11f5dcc68636-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1d671a8c-8e05-4d95-b9bc-50d08a636c4f req-e6bd2bfa-338e-4ec3-98ba-ca6c0710eb1a service nova] Lock "06481254-722a-4053-a6af-11f5dcc68636-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-1d671a8c-8e05-4d95-b9bc-50d08a636c4f req-e6bd2bfa-338e-4ec3-98ba-ca6c0710eb1a service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] No waiting events found dispatching network-vif-plugged-5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:41:26 user nova-compute[72081]: WARNING nova.compute.manager [req-1d671a8c-8e05-4d95-b9bc-50d08a636c4f req-e6bd2bfa-338e-4ec3-98ba-ca6c0710eb1a service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Received unexpected event network-vif-plugged-5861aafc-4312-4d82-965c-2d1d993d8ea0 for instance with vm_state shelved_offloaded and task_state spawning.
Apr 12 03:41:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:28 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270888.1776097, 06481254-722a-4053-a6af-11f5dcc68636 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:41:28 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 06481254-722a-4053-a6af-11f5dcc68636] VM Resumed (Lifecycle Event)
Apr 12 03:41:28 user nova-compute[72081]: DEBUG nova.compute.manager [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:41:28 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:41:28 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Instance spawned successfully.
Apr 12 03:41:28 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:41:28 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: shelved_offloaded, current task_state: spawning, current DB power_state: 4, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:41:28 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 06481254-722a-4053-a6af-11f5dcc68636] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:41:28 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270888.1815856, 06481254-722a-4053-a6af-11f5dcc68636 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:41:28 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 06481254-722a-4053-a6af-11f5dcc68636] VM Started (Lifecycle Event)
Apr 12 03:41:28 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:41:28 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Synchronizing instance power state after lifecycle event "Started"; current vm_state: shelved_offloaded, current task_state: spawning, current DB power_state: 4, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:41:28 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 06481254-722a-4053-a6af-11f5dcc68636] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:41:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-bf1e65c6-6e8d-4407-9cdd-eedb4ae755a9 req-cab67a73-bf82-4cd0-897d-13152ad7b6ed service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Received event network-vif-plugged-5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:41:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bf1e65c6-6e8d-4407-9cdd-eedb4ae755a9 req-cab67a73-bf82-4cd0-897d-13152ad7b6ed service nova] Acquiring lock "06481254-722a-4053-a6af-11f5dcc68636-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bf1e65c6-6e8d-4407-9cdd-eedb4ae755a9 req-cab67a73-bf82-4cd0-897d-13152ad7b6ed service nova] Lock "06481254-722a-4053-a6af-11f5dcc68636-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bf1e65c6-6e8d-4407-9cdd-eedb4ae755a9 req-cab67a73-bf82-4cd0-897d-13152ad7b6ed service nova] Lock "06481254-722a-4053-a6af-11f5dcc68636-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-bf1e65c6-6e8d-4407-9cdd-eedb4ae755a9 req-cab67a73-bf82-4cd0-897d-13152ad7b6ed service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] No waiting events found dispatching network-vif-plugged-5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:41:28 user nova-compute[72081]: WARNING nova.compute.manager [req-bf1e65c6-6e8d-4407-9cdd-eedb4ae755a9 req-cab67a73-bf82-4cd0-897d-13152ad7b6ed service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Received unexpected event network-vif-plugged-5861aafc-4312-4d82-965c-2d1d993d8ea0 for instance with vm_state shelved_offloaded and task_state spawning.
Apr 12 03:41:28 user nova-compute[72081]: DEBUG nova.compute.manager [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:41:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-384b09cc-37df-4279-9da7-c7db26a21ded tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "06481254-722a-4053-a6af-11f5dcc68636" "released" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: held 10.530s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Acquiring lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:32 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:41:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:33 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:41:33 user nova-compute[72081]: INFO nova.compute.claims [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Claim successful on node user
Apr 12 03:41:33 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:41:33 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:41:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.406s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:41:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:41:33 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:41:33 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:41:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:41:33 user nova-compute[72081]: DEBUG nova.policy [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '45fe6fed1a5a43e5bd0925b92486ec68', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4f984af8c7048c7ba68c1530be9cc8d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:41:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:41:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:41:33 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Creating image(s)
Apr 12 03:41:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Acquiring lock "/opt/stack/data/nova/instances/f88ecf99-d158-4f3c-8a87-6c0322fdc8ab/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "/opt/stack/data/nova/instances/f88ecf99-d158-4f3c-8a87-6c0322fdc8ab/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "/opt/stack/data/nova/instances/f88ecf99-d158-4f3c-8a87-6c0322fdc8ab/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.157s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.143s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/f88ecf99-d158-4f3c-8a87-6c0322fdc8ab/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/f88ecf99-d158-4f3c-8a87-6c0322fdc8ab/disk 1073741824" returned: 0 in 0.054s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.204s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:34 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Checking if we can resize image /opt/stack/data/nova/instances/f88ecf99-d158-4f3c-8a87-6c0322fdc8ab/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:41:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f88ecf99-d158-4f3c-8a87-6c0322fdc8ab/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f88ecf99-d158-4f3c-8a87-6c0322fdc8ab/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:34 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Cannot resize image /opt/stack/data/nova/instances/f88ecf99-d158-4f3c-8a87-6c0322fdc8ab/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:41:34 user nova-compute[72081]: DEBUG nova.objects.instance [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lazy-loading 'migration_context' on Instance uuid f88ecf99-d158-4f3c-8a87-6c0322fdc8ab {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:41:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:41:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Ensure instance console log exists: /opt/stack/data/nova/instances/f88ecf99-d158-4f3c-8a87-6c0322fdc8ab/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:41:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:34 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Successfully created port: 1c1d30d9-7f27-4302-bccb-f30b9bf2a36f {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:41:35 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Successfully updated port: 1c1d30d9-7f27-4302-bccb-f30b9bf2a36f {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:41:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Acquiring lock "refresh_cache-f88ecf99-d158-4f3c-8a87-6c0322fdc8ab" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:41:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Acquired lock "refresh_cache-f88ecf99-d158-4f3c-8a87-6c0322fdc8ab" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:41:35 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:41:35 user nova-compute[72081]: DEBUG nova.compute.manager [req-33127330-d8aa-4f1c-8701-a0f76aa798ba req-2e6ca5e2-939f-4b43-a71b-ed3885a130ec service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Received event network-changed-1c1d30d9-7f27-4302-bccb-f30b9bf2a36f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:41:35 user nova-compute[72081]: DEBUG nova.compute.manager [req-33127330-d8aa-4f1c-8701-a0f76aa798ba req-2e6ca5e2-939f-4b43-a71b-ed3885a130ec service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Refreshing instance network info cache due to event network-changed-1c1d30d9-7f27-4302-bccb-f30b9bf2a36f. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:41:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-33127330-d8aa-4f1c-8701-a0f76aa798ba req-2e6ca5e2-939f-4b43-a71b-ed3885a130ec service nova] Acquiring lock "refresh_cache-f88ecf99-d158-4f3c-8a87-6c0322fdc8ab" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:41:35 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:41:35 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Updating instance_info_cache with network_info: [{"id": "1c1d30d9-7f27-4302-bccb-f30b9bf2a36f", "address": "fa:16:3e:74:3e:6a", "network": {"id": "3ec59075-3409-4372-a0ca-6283a5d56ca0", "bridge": "br-int", "label": "tempest-VolumesActionsTest-959291386-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f4f984af8c7048c7ba68c1530be9cc8d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1c1d30d9-7f", "ovs_interfaceid": "1c1d30d9-7f27-4302-bccb-f30b9bf2a36f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:41:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Releasing lock "refresh_cache-f88ecf99-d158-4f3c-8a87-6c0322fdc8ab" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:41:35 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Instance network_info: |[{"id": "1c1d30d9-7f27-4302-bccb-f30b9bf2a36f", "address": "fa:16:3e:74:3e:6a", "network": {"id": "3ec59075-3409-4372-a0ca-6283a5d56ca0", "bridge": "br-int", "label": "tempest-VolumesActionsTest-959291386-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f4f984af8c7048c7ba68c1530be9cc8d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1c1d30d9-7f", "ovs_interfaceid": "1c1d30d9-7f27-4302-bccb-f30b9bf2a36f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:41:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-33127330-d8aa-4f1c-8701-a0f76aa798ba req-2e6ca5e2-939f-4b43-a71b-ed3885a130ec service nova] Acquired lock "refresh_cache-f88ecf99-d158-4f3c-8a87-6c0322fdc8ab" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:41:35 user nova-compute[72081]: DEBUG nova.network.neutron [req-33127330-d8aa-4f1c-8701-a0f76aa798ba req-2e6ca5e2-939f-4b43-a71b-ed3885a130ec service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Refreshing network info cache for port 1c1d30d9-7f27-4302-bccb-f30b9bf2a36f {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:41:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Start _get_guest_xml network_info=[{"id": "1c1d30d9-7f27-4302-bccb-f30b9bf2a36f", "address": "fa:16:3e:74:3e:6a", "network": {"id": "3ec59075-3409-4372-a0ca-6283a5d56ca0", "bridge": "br-int", "label": "tempest-VolumesActionsTest-959291386-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f4f984af8c7048c7ba68c1530be9cc8d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1c1d30d9-7f", "ovs_interfaceid": "1c1d30d9-7f27-4302-bccb-f30b9bf2a36f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:41:36 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:41:36 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:41:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:41:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-1538366054',display_name='tempest-VolumesActionsTest-instance-1538366054',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-1538366054',id=29,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f4f984af8c7048c7ba68c1530be9cc8d',ramdisk_id='',reservation_id='r-gkg6fino',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-629816748',owner_user_name='tempest-VolumesActionsTest-629816748-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:41:34Z,user_data=None,user_id='45fe6fed1a5a43e5bd0925b92486ec68',uuid=f88ecf99-d158-4f3c-8a87-6c0322fdc8ab,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "1c1d30d9-7f27-4302-bccb-f30b9bf2a36f", "address": "fa:16:3e:74:3e:6a", "network": {"id": "3ec59075-3409-4372-a0ca-6283a5d56ca0", "bridge": "br-int", "label": "tempest-VolumesActionsTest-959291386-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f4f984af8c7048c7ba68c1530be9cc8d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1c1d30d9-7f", "ovs_interfaceid": "1c1d30d9-7f27-4302-bccb-f30b9bf2a36f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Converting VIF {"id": "1c1d30d9-7f27-4302-bccb-f30b9bf2a36f", "address": "fa:16:3e:74:3e:6a", "network": {"id": "3ec59075-3409-4372-a0ca-6283a5d56ca0", "bridge": "br-int", "label": "tempest-VolumesActionsTest-959291386-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f4f984af8c7048c7ba68c1530be9cc8d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1c1d30d9-7f", "ovs_interfaceid": "1c1d30d9-7f27-4302-bccb-f30b9bf2a36f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:74:3e:6a,bridge_name='br-int',has_traffic_filtering=True,id=1c1d30d9-7f27-4302-bccb-f30b9bf2a36f,network=Network(3ec59075-3409-4372-a0ca-6283a5d56ca0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1c1d30d9-7f') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG nova.objects.instance [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lazy-loading 'pci_devices' on Instance uuid f88ecf99-d158-4f3c-8a87-6c0322fdc8ab {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:41:36 user nova-compute[72081]:   <uuid>f88ecf99-d158-4f3c-8a87-6c0322fdc8ab</uuid>
Apr 12 03:41:36 user nova-compute[72081]:   <name>instance-0000001d</name>
Apr 12 03:41:36 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:41:36 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:41:36 user nova-compute[72081]:   <metadata>
Apr 12 03:41:36 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:41:36 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:41:36 user nova-compute[72081]:       <nova:name>tempest-VolumesActionsTest-instance-1538366054</nova:name>
Apr 12 03:41:36 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:41:36</nova:creationTime>
Apr 12 03:41:36 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:41:36 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:41:36 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:41:36 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:41:36 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:41:36 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:41:36 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:41:36 user nova-compute[72081]:       <nova:owner>
Apr 12 03:41:36 user nova-compute[72081]:         <nova:user uuid="45fe6fed1a5a43e5bd0925b92486ec68">tempest-VolumesActionsTest-629816748-project-member</nova:user>
Apr 12 03:41:36 user nova-compute[72081]:         <nova:project uuid="f4f984af8c7048c7ba68c1530be9cc8d">tempest-VolumesActionsTest-629816748</nova:project>
Apr 12 03:41:36 user nova-compute[72081]:       </nova:owner>
Apr 12 03:41:36 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:41:36 user nova-compute[72081]:       <nova:ports>
Apr 12 03:41:36 user nova-compute[72081]:         <nova:port uuid="1c1d30d9-7f27-4302-bccb-f30b9bf2a36f">
Apr 12 03:41:36 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Apr 12 03:41:36 user nova-compute[72081]:         </nova:port>
Apr 12 03:41:36 user nova-compute[72081]:       </nova:ports>
Apr 12 03:41:36 user nova-compute[72081]:     </nova:instance>
Apr 12 03:41:36 user nova-compute[72081]:   </metadata>
Apr 12 03:41:36 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:41:36 user nova-compute[72081]:     <system>
Apr 12 03:41:36 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:41:36 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:41:36 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:41:36 user nova-compute[72081]:       <entry name="serial">f88ecf99-d158-4f3c-8a87-6c0322fdc8ab</entry>
Apr 12 03:41:36 user nova-compute[72081]:       <entry name="uuid">f88ecf99-d158-4f3c-8a87-6c0322fdc8ab</entry>
Apr 12 03:41:36 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:41:36 user nova-compute[72081]:     </system>
Apr 12 03:41:36 user nova-compute[72081]:   </sysinfo>
Apr 12 03:41:36 user nova-compute[72081]:   <os>
Apr 12 03:41:36 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:41:36 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:41:36 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:41:36 user nova-compute[72081]:   </os>
Apr 12 03:41:36 user nova-compute[72081]:   <features>
Apr 12 03:41:36 user nova-compute[72081]:     <acpi/>
Apr 12 03:41:36 user nova-compute[72081]:     <apic/>
Apr 12 03:41:36 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:41:36 user nova-compute[72081]:   </features>
Apr 12 03:41:36 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:41:36 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:41:36 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:41:36 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:41:36 user nova-compute[72081]:   </clock>
Apr 12 03:41:36 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:41:36 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:41:36 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:41:36 user nova-compute[72081]:   </cpu>
Apr 12 03:41:36 user nova-compute[72081]:   <devices>
Apr 12 03:41:36 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:41:36 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:41:36 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/f88ecf99-d158-4f3c-8a87-6c0322fdc8ab/disk"/>
Apr 12 03:41:36 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:41:36 user nova-compute[72081]:     </disk>
Apr 12 03:41:36 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:41:36 user nova-compute[72081]:       <mac address="fa:16:3e:74:3e:6a"/>
Apr 12 03:41:36 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:41:36 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:41:36 user nova-compute[72081]:       <target dev="tap1c1d30d9-7f"/>
Apr 12 03:41:36 user nova-compute[72081]:     </interface>
Apr 12 03:41:36 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:41:36 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/f88ecf99-d158-4f3c-8a87-6c0322fdc8ab/console.log" append="off"/>
Apr 12 03:41:36 user nova-compute[72081]:     </serial>
Apr 12 03:41:36 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:41:36 user nova-compute[72081]:     <video>
Apr 12 03:41:36 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:41:36 user nova-compute[72081]:     </video>
Apr 12 03:41:36 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:41:36 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:41:36 user nova-compute[72081]:     </rng>
Apr 12 03:41:36 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:41:36 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:41:36 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:41:36 user nova-compute[72081]:     </memballoon>
Apr 12 03:41:36 user nova-compute[72081]:   </devices>
Apr 12 03:41:36 user nova-compute[72081]: </domain>
Apr 12 03:41:36 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:41:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-1538366054',display_name='tempest-VolumesActionsTest-instance-1538366054',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-1538366054',id=29,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f4f984af8c7048c7ba68c1530be9cc8d',ramdisk_id='',reservation_id='r-gkg6fino',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-629816748',owner_user_name='tempest-VolumesActionsTest-629816748-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:41:34Z,user_data=None,user_id='45fe6fed1a5a43e5bd0925b92486ec68',uuid=f88ecf99-d158-4f3c-8a87-6c0322fdc8ab,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "1c1d30d9-7f27-4302-bccb-f30b9bf2a36f", "address": "fa:16:3e:74:3e:6a", "network": {"id": "3ec59075-3409-4372-a0ca-6283a5d56ca0", "bridge": "br-int", "label": "tempest-VolumesActionsTest-959291386-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f4f984af8c7048c7ba68c1530be9cc8d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1c1d30d9-7f", "ovs_interfaceid": "1c1d30d9-7f27-4302-bccb-f30b9bf2a36f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Converting VIF {"id": "1c1d30d9-7f27-4302-bccb-f30b9bf2a36f", "address": "fa:16:3e:74:3e:6a", "network": {"id": "3ec59075-3409-4372-a0ca-6283a5d56ca0", "bridge": "br-int", "label": "tempest-VolumesActionsTest-959291386-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f4f984af8c7048c7ba68c1530be9cc8d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1c1d30d9-7f", "ovs_interfaceid": "1c1d30d9-7f27-4302-bccb-f30b9bf2a36f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:74:3e:6a,bridge_name='br-int',has_traffic_filtering=True,id=1c1d30d9-7f27-4302-bccb-f30b9bf2a36f,network=Network(3ec59075-3409-4372-a0ca-6283a5d56ca0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1c1d30d9-7f') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG os_vif [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:74:3e:6a,bridge_name='br-int',has_traffic_filtering=True,id=1c1d30d9-7f27-4302-bccb-f30b9bf2a36f,network=Network(3ec59075-3409-4372-a0ca-6283a5d56ca0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1c1d30d9-7f') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap1c1d30d9-7f, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap1c1d30d9-7f, col_values=(('external_ids', {'iface-id': '1c1d30d9-7f27-4302-bccb-f30b9bf2a36f', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:74:3e:6a', 'vm-uuid': 'f88ecf99-d158-4f3c-8a87-6c0322fdc8ab'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:36 user nova-compute[72081]: INFO os_vif [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:74:3e:6a,bridge_name='br-int',has_traffic_filtering=True,id=1c1d30d9-7f27-4302-bccb-f30b9bf2a36f,network=Network(3ec59075-3409-4372-a0ca-6283a5d56ca0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1c1d30d9-7f')
Apr 12 03:41:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] No VIF found with MAC fa:16:3e:74:3e:6a, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG nova.network.neutron [req-33127330-d8aa-4f1c-8701-a0f76aa798ba req-2e6ca5e2-939f-4b43-a71b-ed3885a130ec service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Updated VIF entry in instance network info cache for port 1c1d30d9-7f27-4302-bccb-f30b9bf2a36f. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG nova.network.neutron [req-33127330-d8aa-4f1c-8701-a0f76aa798ba req-2e6ca5e2-939f-4b43-a71b-ed3885a130ec service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Updating instance_info_cache with network_info: [{"id": "1c1d30d9-7f27-4302-bccb-f30b9bf2a36f", "address": "fa:16:3e:74:3e:6a", "network": {"id": "3ec59075-3409-4372-a0ca-6283a5d56ca0", "bridge": "br-int", "label": "tempest-VolumesActionsTest-959291386-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f4f984af8c7048c7ba68c1530be9cc8d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1c1d30d9-7f", "ovs_interfaceid": "1c1d30d9-7f27-4302-bccb-f30b9bf2a36f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:41:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-33127330-d8aa-4f1c-8701-a0f76aa798ba req-2e6ca5e2-939f-4b43-a71b-ed3885a130ec service nova] Releasing lock "refresh_cache-f88ecf99-d158-4f3c-8a87-6c0322fdc8ab" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:41:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-c37568ee-440b-48e8-af85-83ba290ac95e req-3d85a133-88d9-4ec7-8e4c-f907a8d4c4c3 service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Received event network-vif-plugged-1c1d30d9-7f27-4302-bccb-f30b9bf2a36f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:41:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c37568ee-440b-48e8-af85-83ba290ac95e req-3d85a133-88d9-4ec7-8e4c-f907a8d4c4c3 service nova] Acquiring lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c37568ee-440b-48e8-af85-83ba290ac95e req-3d85a133-88d9-4ec7-8e4c-f907a8d4c4c3 service nova] Lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c37568ee-440b-48e8-af85-83ba290ac95e req-3d85a133-88d9-4ec7-8e4c-f907a8d4c4c3 service nova] Lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-c37568ee-440b-48e8-af85-83ba290ac95e req-3d85a133-88d9-4ec7-8e4c-f907a8d4c4c3 service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] No waiting events found dispatching network-vif-plugged-1c1d30d9-7f27-4302-bccb-f30b9bf2a36f {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:41:37 user nova-compute[72081]: WARNING nova.compute.manager [req-c37568ee-440b-48e8-af85-83ba290ac95e req-3d85a133-88d9-4ec7-8e4c-f907a8d4c4c3 service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Received unexpected event network-vif-plugged-1c1d30d9-7f27-4302-bccb-f30b9bf2a36f for instance with vm_state building and task_state spawning.
Apr 12 03:41:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "d211dca2-af04-4dd3-b747-cc3bbd8d310b" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "d211dca2-af04-4dd3-b747-cc3bbd8d310b" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "d211dca2-af04-4dd3-b747-cc3bbd8d310b-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "d211dca2-af04-4dd3-b747-cc3bbd8d310b-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "d211dca2-af04-4dd3-b747-cc3bbd8d310b-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:37 user nova-compute[72081]: INFO nova.compute.manager [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Terminating instance
Apr 12 03:41:37 user nova-compute[72081]: DEBUG nova.compute.manager [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:41:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-8a079f44-27bf-4946-9729-6f5485d930f6 req-75d24ab6-1920-4443-a0b5-4abe4811190a service nova] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Received event network-vif-unplugged-db8f85d3-266d-4a25-b809-3f4cf95a54d7 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8a079f44-27bf-4946-9729-6f5485d930f6 req-75d24ab6-1920-4443-a0b5-4abe4811190a service nova] Acquiring lock "d211dca2-af04-4dd3-b747-cc3bbd8d310b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8a079f44-27bf-4946-9729-6f5485d930f6 req-75d24ab6-1920-4443-a0b5-4abe4811190a service nova] Lock "d211dca2-af04-4dd3-b747-cc3bbd8d310b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8a079f44-27bf-4946-9729-6f5485d930f6 req-75d24ab6-1920-4443-a0b5-4abe4811190a service nova] Lock "d211dca2-af04-4dd3-b747-cc3bbd8d310b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-8a079f44-27bf-4946-9729-6f5485d930f6 req-75d24ab6-1920-4443-a0b5-4abe4811190a service nova] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] No waiting events found dispatching network-vif-unplugged-db8f85d3-266d-4a25-b809-3f4cf95a54d7 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-8a079f44-27bf-4946-9729-6f5485d930f6 req-75d24ab6-1920-4443-a0b5-4abe4811190a service nova] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Received event network-vif-unplugged-db8f85d3-266d-4a25-b809-3f4cf95a54d7 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:38 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Instance destroyed successfully.
Apr 12 03:41:38 user nova-compute[72081]: DEBUG nova.objects.instance [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lazy-loading 'resources' on Instance uuid d211dca2-af04-4dd3-b747-cc3bbd8d310b {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:39:46Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-image-snapshot-server-759861809',display_name='tempest-TestVolumeBootPattern-image-snapshot-server-759861809',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-image-snapshot-server-759861809',id=26,image_ref='1ec76e72-2cb8-4d32-80d6-c41e19d43159',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:40:03Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='003688353e2f4b36831c574ebaf65173',ramdisk_id='',reservation_id='r-ooscgxt6',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_bdm_v2='True',image_boot_roles='reader,member',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_project_name='tempest-TestVolumeBootPattern-235043461',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestVolumeBootPattern-235043461-project-member',image_root_device_name='/dev/vda',owner_project_name='tempest-TestVolumeBootPattern-235043461',owner_user_name='tempest-TestVolumeBootPattern-235043461-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:40:04Z,user_data=None,user_id='b00b348933224558aad1cb12921f100f',uuid=d211dca2-af04-4dd3-b747-cc3bbd8d310b,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "db8f85d3-266d-4a25-b809-3f4cf95a54d7", "address": "fa:16:3e:00:ac:f8", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdb8f85d3-26", "ovs_interfaceid": "db8f85d3-266d-4a25-b809-3f4cf95a54d7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converting VIF {"id": "db8f85d3-266d-4a25-b809-3f4cf95a54d7", "address": "fa:16:3e:00:ac:f8", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdb8f85d3-26", "ovs_interfaceid": "db8f85d3-266d-4a25-b809-3f4cf95a54d7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:00:ac:f8,bridge_name='br-int',has_traffic_filtering=True,id=db8f85d3-266d-4a25-b809-3f4cf95a54d7,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdb8f85d3-26') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG os_vif [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:00:ac:f8,bridge_name='br-int',has_traffic_filtering=True,id=db8f85d3-266d-4a25-b809-3f4cf95a54d7,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdb8f85d3-26') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapdb8f85d3-26, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:38 user nova-compute[72081]: INFO os_vif [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:00:ac:f8,bridge_name='br-int',has_traffic_filtering=True,id=db8f85d3-266d-4a25-b809-3f4cf95a54d7,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdb8f85d3-26')
Apr 12 03:41:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HNXGMAWw', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:41:38 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:41:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b4e99f5-57dc-4670-88c1-067cd9f66d0e]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:41:38 user nova-compute[72081]: No targets found.
Apr 12 03:41:38 user nova-compute[72081]: iSNS:
Apr 12 03:41:38 user nova-compute[72081]: No targets found.
Apr 12 03:41:38 user nova-compute[72081]: STATIC:
Apr 12 03:41:38 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf
Apr 12 03:41:38 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:41:38 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:41:38 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:41:38 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:41:38 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:41:38 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff
Apr 12 03:41:38 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:41:38 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:41:38 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6
Apr 12 03:41:38 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:41:38 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:41:38 user nova-compute[72081]: FIRMWARE:
Apr 12 03:41:38 user nova-compute[72081]: No targets found.
Apr 12 03:41:38 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:41:38 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:41:38 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3a5ad14c-108e-400c-a19d-b91d0575981c]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f78f86c9-b365-4c4a-a05c-79aa96d2f115]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:41:38 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:41:38 user nova-compute[72081]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff (non-flash)
Apr 12 03:41:38 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:41:38 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:41:38 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:41:38 user nova-compute[72081]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff (non-flash)
Apr 12 03:41:38 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:41:38 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3f941197e0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff'): ({'sdc'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Removing single pathed devices sdc {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0c6b1d0c-a231-46ec-925b-072b38669f47]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[59e71fbd-bb94-413b-8eb6-bc91a61c4248]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Flushing IO for device /dev/sdc {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7e4445ec-4079-43c2-935f-0360d5444dd6]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.046s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[763dbdd3-d284-4744-8d0c-10db1a88481e]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] SCSI volumes sdc have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-1SCST_BIO2f3474da-8bce8f4733938d14', '/dev/disk/by-id/scsi-23862636538663437', '/dev/disk/by-id/scsi-SSCST_BIO_Storage_2f3474da') {{(pid=82083) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[cbdfc00e-57ba-4093-b0a1-4292522ef4ed]: (4, None) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f4ff1849-b1da-4e4d-a7ab-dc8cc6d35299]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff -p 172.16.0.220:3260 --logout" returned: 0 in 0.074s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a00bdcdb-d4f0-455b-9910-ebe8d0ec6d7d]: (4, ('Logging out of session [sid: 26, target: iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff, portal: 172.16.0.220,3260]\nLogout of [sid: 26, target: iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 26, target: iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff, portal: 172.16.0.220,3260]
Apr 12 03:41:38 user nova-compute[72081]: Logout of [sid: 26, target: iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff, portal: 172.16.0.220,3260] successful.
Apr 12 03:41:38 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43533501-0b6e-4f4e-a56e-53944664ffff -p 172.16.0.220:3260 --op delete" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1bba1391-de9d-4ab5-b59c-80282f930e7f]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.344s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] <== disconnect_volume: return (345ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:41:38 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Deleting instance files /opt/stack/data/nova/instances/d211dca2-af04-4dd3-b747-cc3bbd8d310b_del
Apr 12 03:41:38 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Deletion of /opt/stack/data/nova/instances/d211dca2-af04-4dd3-b747-cc3bbd8d310b_del complete
Apr 12 03:41:38 user nova-compute[72081]: INFO nova.compute.manager [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Took 1.18 seconds to destroy the instance on the hypervisor.
Apr 12 03:41:38 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:41:38 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:41:39 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270899.6435926, f88ecf99-d158-4f3c-8a87-6c0322fdc8ab => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:41:39 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] VM Resumed (Lifecycle Event)
Apr 12 03:41:39 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:41:39 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:41:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:41:39 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Instance spawned successfully.
Apr 12 03:41:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:41:39 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:41:39 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Took 0.70 seconds to deallocate network for instance.
Apr 12 03:41:39 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:41:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:41:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:41:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:41:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:41:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:41:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:41:39 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:41:39 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270899.6448174, f88ecf99-d158-4f3c-8a87-6c0322fdc8ab => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:41:39 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] VM Started (Lifecycle Event)
Apr 12 03:41:39 user nova-compute[72081]: DEBUG nova.compute.manager [req-e0d1937c-10a7-4033-ae53-94dcbdceee72 req-3120b1d0-1ee6-4469-9480-34d8816b9188 service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Received event network-vif-plugged-1c1d30d9-7f27-4302-bccb-f30b9bf2a36f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:41:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e0d1937c-10a7-4033-ae53-94dcbdceee72 req-3120b1d0-1ee6-4469-9480-34d8816b9188 service nova] Acquiring lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e0d1937c-10a7-4033-ae53-94dcbdceee72 req-3120b1d0-1ee6-4469-9480-34d8816b9188 service nova] Lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e0d1937c-10a7-4033-ae53-94dcbdceee72 req-3120b1d0-1ee6-4469-9480-34d8816b9188 service nova] Lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:39 user nova-compute[72081]: DEBUG nova.compute.manager [req-e0d1937c-10a7-4033-ae53-94dcbdceee72 req-3120b1d0-1ee6-4469-9480-34d8816b9188 service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] No waiting events found dispatching network-vif-plugged-1c1d30d9-7f27-4302-bccb-f30b9bf2a36f {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:41:39 user nova-compute[72081]: WARNING nova.compute.manager [req-e0d1937c-10a7-4033-ae53-94dcbdceee72 req-3120b1d0-1ee6-4469-9480-34d8816b9188 service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Received unexpected event network-vif-plugged-1c1d30d9-7f27-4302-bccb-f30b9bf2a36f for instance with vm_state building and task_state spawning.
Apr 12 03:41:39 user nova-compute[72081]: DEBUG nova.compute.manager [req-e0d1937c-10a7-4033-ae53-94dcbdceee72 req-3120b1d0-1ee6-4469-9480-34d8816b9188 service nova] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Received event network-vif-deleted-db8f85d3-266d-4a25-b809-3f4cf95a54d7 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:41:39 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:41:39 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:41:39 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:41:39 user nova-compute[72081]: INFO nova.compute.manager [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Took 6.02 seconds to spawn the instance on the hypervisor.
Apr 12 03:41:39 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:41:39 user nova-compute[72081]: INFO nova.compute.manager [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Took 6.81 seconds to build instance.
Apr 12 03:41:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8340c1df-e680-4314-bf5b-96d5d23b3712 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.943s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:40 user nova-compute[72081]: DEBUG nova.compute.manager [req-bd6ff5e5-92e4-473c-a4f6-296fd8d151fa req-b3877949-b848-4918-9f22-d7cc85e0afaa service nova] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Received event network-vif-plugged-db8f85d3-266d-4a25-b809-3f4cf95a54d7 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:41:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bd6ff5e5-92e4-473c-a4f6-296fd8d151fa req-b3877949-b848-4918-9f22-d7cc85e0afaa service nova] Acquiring lock "d211dca2-af04-4dd3-b747-cc3bbd8d310b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bd6ff5e5-92e4-473c-a4f6-296fd8d151fa req-b3877949-b848-4918-9f22-d7cc85e0afaa service nova] Lock "d211dca2-af04-4dd3-b747-cc3bbd8d310b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bd6ff5e5-92e4-473c-a4f6-296fd8d151fa req-b3877949-b848-4918-9f22-d7cc85e0afaa service nova] Lock "d211dca2-af04-4dd3-b747-cc3bbd8d310b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:40 user nova-compute[72081]: DEBUG nova.compute.manager [req-bd6ff5e5-92e4-473c-a4f6-296fd8d151fa req-b3877949-b848-4918-9f22-d7cc85e0afaa service nova] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] No waiting events found dispatching network-vif-plugged-db8f85d3-266d-4a25-b809-3f4cf95a54d7 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:41:40 user nova-compute[72081]: WARNING nova.compute.manager [req-bd6ff5e5-92e4-473c-a4f6-296fd8d151fa req-b3877949-b848-4918-9f22-d7cc85e0afaa service nova] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Received unexpected event network-vif-plugged-db8f85d3-266d-4a25-b809-3f4cf95a54d7 for instance with vm_state active and task_state deleting.
Apr 12 03:41:40 user nova-compute[72081]: INFO nova.compute.manager [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Took 0.91 seconds to detach 1 volumes for instance.
Apr 12 03:41:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Deleting volume: 43533501-0b6e-4f4e-a56e-53944664ffff {{(pid=72081) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3202}}
Apr 12 03:41:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:41 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:41:41 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:41:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.449s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:41 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Deleted allocations for instance d211dca2-af04-4dd3-b747-cc3bbd8d310b
Apr 12 03:41:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-23f7af81-6f12-4b49-9115-195dead30880 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "d211dca2-af04-4dd3-b747-cc3bbd8d310b" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.527s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "614495f2-f108-4759-be60-bc6cc689c5b7" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "614495f2-f108-4759-be60-bc6cc689c5b7" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:41 user nova-compute[72081]: DEBUG nova.compute.manager [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:41:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:42 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:41:42 user nova-compute[72081]: INFO nova.compute.claims [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Claim successful on node user
Apr 12 03:41:42 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:41:42 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:41:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.454s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:42 user nova-compute[72081]: DEBUG nova.compute.manager [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:41:42 user nova-compute[72081]: DEBUG nova.compute.manager [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:41:42 user nova-compute[72081]: DEBUG nova.network.neutron [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:41:42 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:41:42 user nova-compute[72081]: DEBUG nova.compute.manager [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:41:42 user nova-compute[72081]: INFO nova.virt.block_device [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Booting with volume-backed-image e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7 at /dev/vda
Apr 12 03:41:42 user nova-compute[72081]: DEBUG nova.policy [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd99be569057f432484e4a60b46882bbd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e1fa352a8c144a1aada7ce0d8818429', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:41:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:43 user nova-compute[72081]: DEBUG nova.network.neutron [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Successfully created port: b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:41:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:44 user nova-compute[72081]: DEBUG nova.network.neutron [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Successfully updated port: b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:41:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "refresh_cache-614495f2-f108-4759-be60-bc6cc689c5b7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:41:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquired lock "refresh_cache-614495f2-f108-4759-be60-bc6cc689c5b7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:41:44 user nova-compute[72081]: DEBUG nova.network.neutron [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:41:44 user nova-compute[72081]: DEBUG nova.compute.manager [req-7b4f0d48-85f4-4627-a433-11cebaa8ee0a req-fb227d8c-a9ed-41c1-9f83-804eaa334f7b service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received event network-changed-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:41:44 user nova-compute[72081]: DEBUG nova.compute.manager [req-7b4f0d48-85f4-4627-a433-11cebaa8ee0a req-fb227d8c-a9ed-41c1-9f83-804eaa334f7b service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Refreshing instance network info cache due to event network-changed-b9acaee6-a684-4827-bc91-1c6219782efe. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:41:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7b4f0d48-85f4-4627-a433-11cebaa8ee0a req-fb227d8c-a9ed-41c1-9f83-804eaa334f7b service nova] Acquiring lock "refresh_cache-614495f2-f108-4759-be60-bc6cc689c5b7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:41:44 user nova-compute[72081]: DEBUG nova.network.neutron [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:41:44 user nova-compute[72081]: DEBUG nova.network.neutron [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Updating instance_info_cache with network_info: [{"id": "b9acaee6-a684-4827-bc91-1c6219782efe", "address": "fa:16:3e:d4:dc:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb9acaee6-a6", "ovs_interfaceid": "b9acaee6-a684-4827-bc91-1c6219782efe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:41:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Releasing lock "refresh_cache-614495f2-f108-4759-be60-bc6cc689c5b7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:41:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Instance network_info: |[{"id": "b9acaee6-a684-4827-bc91-1c6219782efe", "address": "fa:16:3e:d4:dc:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb9acaee6-a6", "ovs_interfaceid": "b9acaee6-a684-4827-bc91-1c6219782efe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:41:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7b4f0d48-85f4-4627-a433-11cebaa8ee0a req-fb227d8c-a9ed-41c1-9f83-804eaa334f7b service nova] Acquired lock "refresh_cache-614495f2-f108-4759-be60-bc6cc689c5b7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:41:44 user nova-compute[72081]: DEBUG nova.network.neutron [req-7b4f0d48-85f4-4627-a433-11cebaa8ee0a req-fb227d8c-a9ed-41c1-9f83-804eaa334f7b service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Refreshing network info cache for port b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:41:45 user nova-compute[72081]: DEBUG nova.network.neutron [req-7b4f0d48-85f4-4627-a433-11cebaa8ee0a req-fb227d8c-a9ed-41c1-9f83-804eaa334f7b service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Updated VIF entry in instance network info cache for port b9acaee6-a684-4827-bc91-1c6219782efe. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:41:45 user nova-compute[72081]: DEBUG nova.network.neutron [req-7b4f0d48-85f4-4627-a433-11cebaa8ee0a req-fb227d8c-a9ed-41c1-9f83-804eaa334f7b service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Updating instance_info_cache with network_info: [{"id": "b9acaee6-a684-4827-bc91-1c6219782efe", "address": "fa:16:3e:d4:dc:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb9acaee6-a6", "ovs_interfaceid": "b9acaee6-a684-4827-bc91-1c6219782efe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:41:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7b4f0d48-85f4-4627-a433-11cebaa8ee0a req-fb227d8c-a9ed-41c1-9f83-804eaa334f7b service nova] Releasing lock "refresh_cache-614495f2-f108-4759-be60-bc6cc689c5b7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:41:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:48 user nova-compute[72081]: DEBUG os_brick.utils [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:41:48 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:48 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:48 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8c8877a1-1faa-4cad-b38c-688ab86e5e98]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:48 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:48 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:48 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[28459fee-5aa7-4d72-8a3e-ab63890de272]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:48 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:48 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:48 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b073d9d5-d679-4597-8609-510465f5da6f]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:48 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[510ed739-0ccc-4c19-8de0-a7cdfe802bfe]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:48 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:48 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:41:48 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:41:48 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:48 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:41:48 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:41:48 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[cfaaa99f-e681-4d70-b8fd-eebb07e864f8]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:48 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:41:48 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:41:48 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:41:48 user nova-compute[72081]: DEBUG os_brick.utils [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] <== get_connector_properties: return (122ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:41:48 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Updating existing volume attachment record: e53160c2-1524-4d29-8668-bda2166b6071 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:41:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG nova.compute.manager [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:41:53 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Creating image(s)
Apr 12 03:41:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Did not create local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Ensure instance console log exists: /opt/stack/data/nova/instances/614495f2-f108-4759-be60-bc6cc689c5b7/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Start _get_guest_xml network_info=[{"id": "b9acaee6-a684-4827-bc91-1c6219782efe", "address": "fa:16:3e:d4:dc:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb9acaee6-a6", "ovs_interfaceid": "b9acaee6-a684-4827-bc91-1c6219782efe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'fbzrSZhe', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '614495f2-f108-4759-be60-bc6cc689c5b7', 'attached_at': '', 'detached_at': '', 'volume_id': '54a9604a-4853-414c-850e-51745d6fcd0e', 'serial': '54a9604a-4853-414c-850e-51745d6fcd0e'}, 'disk_bus': 'virtio', 'delete_on_termination': False, 'boot_index': 0, 'guest_format': None, 'device_type': 'disk', 'attachment_id': 'e53160c2-1524-4d29-8668-bda2166b6071', 'volume_type': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:41:53 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:41:53 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:41:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'fbzrSZhe', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:41:53 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:41:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:41:53 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:41:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:41:53 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:41:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b6f87d42-a1a8-484e-9d63-d6b8945b2f06]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:41:53 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c553b89-2ae2-494e-af14-4ac67cf9c119]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e] added
Apr 12 03:41:53 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[dca6a11a-f55d-4e9c-aa40-49237d8879de]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0bfbe799-6e2f-4774-9232-7acde9dea030]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v fbzrSZhe {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v fbzrSZhe" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[19b0e25c-2aa0-4340-a105-87e42eb79678]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'fbzrSZhe'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4e8f50b9-8976-4960-808c-da03ab8ee4c2]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9e2d758b-2dd8-4c83-afdb-21089a9218b7]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:41:53 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:41:53 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:41:53 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:41:53 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:41:53 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:41:53 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --login" returned: 0 in 0.075s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5f6456a9-81a2-4e76-a879-7678a5d8981c]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e, portal: 172.16.0.220,3260]
Apr 12 03:41:53 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e, portal: 172.16.0.220,3260] successful.
Apr 12 03:41:53 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[55beec92-8e9f-4668-9dfe-e4895a24f4fa]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9a642361-1614-4fce-afc3-453c91318df7]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:41:53 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:41:53 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:41:53 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:41:53 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:41:53 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:41:53 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:41:53 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:41:53 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.312s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] HCTL ('34', '-', '-', 0) found on session 28 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[84cf6776-2fba-4cad-856e-b9ba2300e066]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Searching for a device in session 28 and hctl ['34', '*', '*', 0] yield: None {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681270898.524788, d211dca2-af04-4dd3-b747-cc3bbd8d310b => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:41:53 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] VM Stopped (Lifecycle Event)
Apr 12 03:41:53 user nova-compute[72081]: DEBUG nova.compute.manager [None req-cfb2c70c-a441-42ef-b645-945dc3331dd4 None None] [instance: d211dca2-af04-4dd3-b747-cc3bbd8d310b] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:41:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:54 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Searching for a device in session 28 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:41:54 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'fbzrSZhe', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:41:54 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.341s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:41:54 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] <== connect_volume: return (1343ms) {'type': 'block', 'scsi_wwn': '23037353334373734', 'path': '/dev/sdc'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:41:54 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23037353334373734', 'path': '/dev/sdc'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:41:54 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:41:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1156151605',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1156151605',id=30,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6e1fa352a8c144a1aada7ce0d8818429',ramdisk_id='',reservation_id='r-r2n25ts0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-590302458',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:41:43Z,user_data=None,user_id='d99be569057f432484e4a60b46882bbd',uuid=614495f2-f108-4759-be60-bc6cc689c5b7,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "b9acaee6-a684-4827-bc91-1c6219782efe", "address": "fa:16:3e:d4:dc:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb9acaee6-a6", "ovs_interfaceid": "b9acaee6-a684-4827-bc91-1c6219782efe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:41:54 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converting VIF {"id": "b9acaee6-a684-4827-bc91-1c6219782efe", "address": "fa:16:3e:d4:dc:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb9acaee6-a6", "ovs_interfaceid": "b9acaee6-a684-4827-bc91-1c6219782efe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:41:54 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d4:dc:db,bridge_name='br-int',has_traffic_filtering=True,id=b9acaee6-a684-4827-bc91-1c6219782efe,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb9acaee6-a6') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:41:54 user nova-compute[72081]: DEBUG nova.objects.instance [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lazy-loading 'pci_devices' on Instance uuid 614495f2-f108-4759-be60-bc6cc689c5b7 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:41:54 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:41:54 user nova-compute[72081]:   <uuid>614495f2-f108-4759-be60-bc6cc689c5b7</uuid>
Apr 12 03:41:54 user nova-compute[72081]:   <name>instance-0000001e</name>
Apr 12 03:41:54 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:41:54 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:41:54 user nova-compute[72081]:   <metadata>
Apr 12 03:41:54 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:41:54 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:41:54 user nova-compute[72081]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-1156151605</nova:name>
Apr 12 03:41:54 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:41:53</nova:creationTime>
Apr 12 03:41:54 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:41:54 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:41:54 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:41:54 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:41:54 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:41:54 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:41:54 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:41:54 user nova-compute[72081]:       <nova:owner>
Apr 12 03:41:54 user nova-compute[72081]:         <nova:user uuid="d99be569057f432484e4a60b46882bbd">tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member</nova:user>
Apr 12 03:41:54 user nova-compute[72081]:         <nova:project uuid="6e1fa352a8c144a1aada7ce0d8818429">tempest-ServerBootFromVolumeStableRescueTest-590302458</nova:project>
Apr 12 03:41:54 user nova-compute[72081]:       </nova:owner>
Apr 12 03:41:54 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:41:54 user nova-compute[72081]:       <nova:ports>
Apr 12 03:41:54 user nova-compute[72081]:         <nova:port uuid="b9acaee6-a684-4827-bc91-1c6219782efe">
Apr 12 03:41:54 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.8" ipVersion="4"/>
Apr 12 03:41:54 user nova-compute[72081]:         </nova:port>
Apr 12 03:41:54 user nova-compute[72081]:       </nova:ports>
Apr 12 03:41:54 user nova-compute[72081]:     </nova:instance>
Apr 12 03:41:54 user nova-compute[72081]:   </metadata>
Apr 12 03:41:54 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:41:54 user nova-compute[72081]:     <system>
Apr 12 03:41:54 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:41:54 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:41:54 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:41:54 user nova-compute[72081]:       <entry name="serial">614495f2-f108-4759-be60-bc6cc689c5b7</entry>
Apr 12 03:41:54 user nova-compute[72081]:       <entry name="uuid">614495f2-f108-4759-be60-bc6cc689c5b7</entry>
Apr 12 03:41:54 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:41:54 user nova-compute[72081]:     </system>
Apr 12 03:41:54 user nova-compute[72081]:   </sysinfo>
Apr 12 03:41:54 user nova-compute[72081]:   <os>
Apr 12 03:41:54 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:41:54 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:41:54 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:41:54 user nova-compute[72081]:   </os>
Apr 12 03:41:54 user nova-compute[72081]:   <features>
Apr 12 03:41:54 user nova-compute[72081]:     <acpi/>
Apr 12 03:41:54 user nova-compute[72081]:     <apic/>
Apr 12 03:41:54 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:41:54 user nova-compute[72081]:   </features>
Apr 12 03:41:54 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:41:54 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:41:54 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:41:54 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:41:54 user nova-compute[72081]:   </clock>
Apr 12 03:41:54 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:41:54 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:41:54 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:41:54 user nova-compute[72081]:   </cpu>
Apr 12 03:41:54 user nova-compute[72081]:   <devices>
Apr 12 03:41:54 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:41:54 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:41:54 user nova-compute[72081]:       <source dev="/dev/sdc"/>
Apr 12 03:41:54 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:41:54 user nova-compute[72081]:       <serial>54a9604a-4853-414c-850e-51745d6fcd0e</serial>
Apr 12 03:41:54 user nova-compute[72081]:     </disk>
Apr 12 03:41:54 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:41:54 user nova-compute[72081]:       <mac address="fa:16:3e:d4:dc:db"/>
Apr 12 03:41:54 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:41:54 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:41:54 user nova-compute[72081]:       <target dev="tapb9acaee6-a6"/>
Apr 12 03:41:54 user nova-compute[72081]:     </interface>
Apr 12 03:41:54 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:41:54 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/614495f2-f108-4759-be60-bc6cc689c5b7/console.log" append="off"/>
Apr 12 03:41:54 user nova-compute[72081]:     </serial>
Apr 12 03:41:54 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:41:54 user nova-compute[72081]:     <video>
Apr 12 03:41:54 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:41:54 user nova-compute[72081]:     </video>
Apr 12 03:41:54 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:41:54 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:41:54 user nova-compute[72081]:     </rng>
Apr 12 03:41:54 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:41:54 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:41:54 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:41:54 user nova-compute[72081]:     </memballoon>
Apr 12 03:41:54 user nova-compute[72081]:   </devices>
Apr 12 03:41:54 user nova-compute[72081]: </domain>
Apr 12 03:41:54 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:41:54 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:41:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1156151605',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1156151605',id=30,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6e1fa352a8c144a1aada7ce0d8818429',ramdisk_id='',reservation_id='r-r2n25ts0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-590302458',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:41:43Z,user_data=None,user_id='d99be569057f432484e4a60b46882bbd',uuid=614495f2-f108-4759-be60-bc6cc689c5b7,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "b9acaee6-a684-4827-bc91-1c6219782efe", "address": "fa:16:3e:d4:dc:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb9acaee6-a6", "ovs_interfaceid": "b9acaee6-a684-4827-bc91-1c6219782efe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:41:54 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converting VIF {"id": "b9acaee6-a684-4827-bc91-1c6219782efe", "address": "fa:16:3e:d4:dc:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb9acaee6-a6", "ovs_interfaceid": "b9acaee6-a684-4827-bc91-1c6219782efe", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:41:54 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d4:dc:db,bridge_name='br-int',has_traffic_filtering=True,id=b9acaee6-a684-4827-bc91-1c6219782efe,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb9acaee6-a6') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:41:54 user nova-compute[72081]: DEBUG os_vif [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:d4:dc:db,bridge_name='br-int',has_traffic_filtering=True,id=b9acaee6-a684-4827-bc91-1c6219782efe,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb9acaee6-a6') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:41:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:41:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:41:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapb9acaee6-a6, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:41:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapb9acaee6-a6, col_values=(('external_ids', {'iface-id': 'b9acaee6-a684-4827-bc91-1c6219782efe', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:d4:dc:db', 'vm-uuid': '614495f2-f108-4759-be60-bc6cc689c5b7'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:41:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:41:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:54 user nova-compute[72081]: INFO os_vif [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:d4:dc:db,bridge_name='br-int',has_traffic_filtering=True,id=b9acaee6-a684-4827-bc91-1c6219782efe,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb9acaee6-a6')
Apr 12 03:41:54 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:41:54 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] No VIF found with MAC fa:16:3e:d4:dc:db, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:41:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG nova.compute.manager [req-8c13e802-252d-4bb7-8d0e-1b43c91889c4 req-d2320cd4-8c2f-4eb7-9c91-900235fa3d73 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received event network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8c13e802-252d-4bb7-8d0e-1b43c91889c4 req-d2320cd4-8c2f-4eb7-9c91-900235fa3d73 service nova] Acquiring lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8c13e802-252d-4bb7-8d0e-1b43c91889c4 req-d2320cd4-8c2f-4eb7-9c91-900235fa3d73 service nova] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8c13e802-252d-4bb7-8d0e-1b43c91889c4 req-d2320cd4-8c2f-4eb7-9c91-900235fa3d73 service nova] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG nova.compute.manager [req-8c13e802-252d-4bb7-8d0e-1b43c91889c4 req-d2320cd4-8c2f-4eb7-9c91-900235fa3d73 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] No waiting events found dispatching network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:41:56 user nova-compute[72081]: WARNING nova.compute.manager [req-8c13e802-252d-4bb7-8d0e-1b43c91889c4 req-d2320cd4-8c2f-4eb7-9c91-900235fa3d73 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received unexpected event network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe for instance with vm_state building and task_state spawning.
Apr 12 03:41:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG nova.objects.instance [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'flavor' on Instance uuid 48977eb2-ce43-4577-a1cb-340e35703dbd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.050s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:56 user nova-compute[72081]: INFO nova.compute.manager [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Attaching volume 8278d82d-cc19-4660-b4f1-b13d92e7d742 to /dev/vdb
Apr 12 03:41:56 user nova-compute[72081]: DEBUG os_brick.utils [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9ee8543d-3c6b-4b16-8310-8d16aad03854]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f779312-f84d-4660-aa32-ca9face8e5e3]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4be8e2ba-76de-4910-b34c-b080cdb33674]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9b5c82a0-51a2-4286-9d27-d6c0e14e92f5]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:41:56 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:41:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[24bdd88a-6209-4853-9f9a-3575c012922a]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG os_brick.utils [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] <== get_connector_properties: return (123ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:41:56 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Updating existing volume attachment record: b6e551d2-39d5-479e-a3c5-f1e4283444a0 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:41:57 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270917.9562483, 614495f2-f108-4759-be60-bc6cc689c5b7 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:41:57 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] VM Resumed (Lifecycle Event)
Apr 12 03:41:57 user nova-compute[72081]: DEBUG nova.compute.manager [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:41:57 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:41:57 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Instance spawned successfully.
Apr 12 03:41:57 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:41:57 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:41:58 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:41:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:41:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:41:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:41:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:41:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:41:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:41:58 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:41:58 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270917.9664552, 614495f2-f108-4759-be60-bc6cc689c5b7 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:41:58 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] VM Started (Lifecycle Event)
Apr 12 03:41:58 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:41:58 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:41:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:58 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:41:58 user nova-compute[72081]: INFO nova.compute.manager [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Took 5.10 seconds to spawn the instance on the hypervisor.
Apr 12 03:41:58 user nova-compute[72081]: DEBUG nova.compute.manager [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:41:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-f9c7653b-c0fd-4e80-a132-7f60bbfcd2d8 req-2a61325e-f2b9-4685-aecc-f02559d7e045 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received event network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:41:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f9c7653b-c0fd-4e80-a132-7f60bbfcd2d8 req-2a61325e-f2b9-4685-aecc-f02559d7e045 service nova] Acquiring lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:41:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f9c7653b-c0fd-4e80-a132-7f60bbfcd2d8 req-2a61325e-f2b9-4685-aecc-f02559d7e045 service nova] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:41:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f9c7653b-c0fd-4e80-a132-7f60bbfcd2d8 req-2a61325e-f2b9-4685-aecc-f02559d7e045 service nova] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-f9c7653b-c0fd-4e80-a132-7f60bbfcd2d8 req-2a61325e-f2b9-4685-aecc-f02559d7e045 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] No waiting events found dispatching network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:41:58 user nova-compute[72081]: WARNING nova.compute.manager [req-f9c7653b-c0fd-4e80-a132-7f60bbfcd2d8 req-2a61325e-f2b9-4685-aecc-f02559d7e045 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received unexpected event network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe for instance with vm_state building and task_state spawning.
Apr 12 03:41:58 user nova-compute[72081]: INFO nova.compute.manager [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Took 16.25 seconds to build instance.
Apr 12 03:41:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-58209f43-4f79-4263-b69f-9ac01062ee7d tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "614495f2-f108-4759-be60-bc6cc689c5b7" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 16.351s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:41:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:41:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YDhqrwbV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:42:01 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:42:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:42:01 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:42:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:42:01 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[18f86d8d-d4b5-4fb4-bcef-ea298a51aaf6]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:42:01 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4f9e0518-2505-49bc-901f-0f2807ff5069]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742] added
Apr 12 03:42:01 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.028s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[22935b62-2717-48a9-829d-c93e38c5309a]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d1ff0d7a-c618-40ec-af95-a39dd49b41c2]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YDhqrwbV {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YDhqrwbV" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7779c1f9-fd3e-4859-9aed-1b75e72e7c2a]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'YDhqrwbV'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3ca1d167-fa54-4e01-a0b4-cd65396b7125]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[86c8ba21-0249-41d1-b371-128fa3eaee4b]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:42:01 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:42:01 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:42:01 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:42:01 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:42:01 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:42:01 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:42:01 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:42:01 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --login" returned: 0 in 0.071s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4fc087b2-fe16-4840-b9d7-3b08e850e667]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742, portal: 172.16.0.220,3260]
Apr 12 03:42:01 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742, portal: 172.16.0.220,3260] successful.
Apr 12 03:42:01 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[acf06794-3df0-46a6-b90e-1af946f70ad8]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[44f0b045-0b79-4634-a8f9-6edede326d87]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:42:01 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:42:01 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:42:01 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:42:01 user nova-compute[72081]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:42:01 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:42:01 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:42:01 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:42:01 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:42:01 user nova-compute[72081]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:42:01 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.320s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] HCTL ('37', '-', '-', 0) found on session 29 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Scanning host 37 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[19befe7c-3e72-4041-a967-8a05dc02485a]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Searching for a device in session 29 and hctl ['37', '*', '*', 0] yield: sdf {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:42:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Connected to sdf using {'target_iqn': 'iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YDhqrwbV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:42:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:02 user nova-compute[72081]: DEBUG nova.compute.manager [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:42:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:02 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:42:02 user nova-compute[72081]: INFO nova.compute.claims [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Claim successful on node user
Apr 12 03:42:02 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.361s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:42:02 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] <== connect_volume: return (1362ms) {'type': 'block', 'scsi_wwn': '26437356130623139', 'path': '/dev/sdf'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:42:02 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26437356130623139', 'path': '/dev/sdf'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:42:02 user nova-compute[72081]: DEBUG nova.objects.instance [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'flavor' on Instance uuid 48977eb2-ce43-4577-a1cb-340e35703dbd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:42:02 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] attach device xml: <disk type="block" device="disk">
Apr 12 03:42:02 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:42:02 user nova-compute[72081]:   <source dev="/dev/sdf"/>
Apr 12 03:42:02 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:42:02 user nova-compute[72081]:   <serial>8278d82d-cc19-4660-b4f1-b13d92e7d742</serial>
Apr 12 03:42:02 user nova-compute[72081]: </disk>
Apr 12 03:42:02 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 03:42:02 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:42:02 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.448s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG nova.network.neutron [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:03 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:42:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] No VIF found with MAC fa:16:3e:b3:c5:ca, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG nova.policy [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '872fbc3d59274cd480c17b4b7342499a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a341e582192a47548df8627d82c96ea8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:42:03 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Creating image(s)
Apr 12 03:42:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "/opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "/opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "/opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-30fb675a-f318-4c7a-9d68-849294e7d1ea tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.665s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.158s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk 1073741824" returned: 0 in 0.049s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.191s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.155s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Checking if we can resize image /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Cannot resize image /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG nova.objects.instance [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'migration_context' on Instance uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Ensure instance console log exists: /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:04 user nova-compute[72081]: DEBUG nova.network.neutron [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Successfully created port: e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:42:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:04 user nova-compute[72081]: DEBUG nova.network.neutron [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Successfully updated port: e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:42:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "refresh_cache-606425bb-8968-41f9-b3a5-80b6fd188a67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:42:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquired lock "refresh_cache-606425bb-8968-41f9-b3a5-80b6fd188a67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:42:04 user nova-compute[72081]: DEBUG nova.network.neutron [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:42:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-ae2a6615-d067-4aec-9d8c-803e68d98f9d req-f656e22c-5738-4e45-b154-3b8c4858f0f7 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received event network-changed-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:42:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-ae2a6615-d067-4aec-9d8c-803e68d98f9d req-f656e22c-5738-4e45-b154-3b8c4858f0f7 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Refreshing instance network info cache due to event network-changed-e16af111-f1d4-4f8c-a67a-c80f546c37b1. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:42:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ae2a6615-d067-4aec-9d8c-803e68d98f9d req-f656e22c-5738-4e45-b154-3b8c4858f0f7 service nova] Acquiring lock "refresh_cache-606425bb-8968-41f9-b3a5-80b6fd188a67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:42:04 user nova-compute[72081]: DEBUG nova.network.neutron [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.network.neutron [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Updating instance_info_cache with network_info: [{"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Releasing lock "refresh_cache-606425bb-8968-41f9-b3a5-80b6fd188a67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.compute.manager [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Instance network_info: |[{"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ae2a6615-d067-4aec-9d8c-803e68d98f9d req-f656e22c-5738-4e45-b154-3b8c4858f0f7 service nova] Acquired lock "refresh_cache-606425bb-8968-41f9-b3a5-80b6fd188a67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.network.neutron [req-ae2a6615-d067-4aec-9d8c-803e68d98f9d req-f656e22c-5738-4e45-b154-3b8c4858f0f7 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Refreshing network info cache for port e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Start _get_guest_xml network_info=[{"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:42:05 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:42:05 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:42:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-2123178219',display_name='tempest-AttachVolumeTestJSON-server-2123178219',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-2123178219',id=31,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDsSrynzjRvdumscunYM7qq2LINQa1dLMdBCKJ6XwNU+wH30BpZ+UxgkL98cPRjcfl2uMCQ+6nM9+XV2FMWC/Uf6P+0R1p0Yeexa689yM1BKJkKUWaai92yYlAVXCdH/1Q==',key_name='tempest-keypair-611227490',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a341e582192a47548df8627d82c96ea8',ramdisk_id='',reservation_id='r-qag42n01',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeTestJSON-617614045',owner_user_name='tempest-AttachVolumeTestJSON-617614045-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:42:03Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='872fbc3d59274cd480c17b4b7342499a',uuid=606425bb-8968-41f9-b3a5-80b6fd188a67,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Converting VIF {"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:85:88:9b,bridge_name='br-int',has_traffic_filtering=True,id=e16af111-f1d4-4f8c-a67a-c80f546c37b1,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16af111-f1') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.objects.instance [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'pci_devices' on Instance uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:42:05 user nova-compute[72081]:   <uuid>606425bb-8968-41f9-b3a5-80b6fd188a67</uuid>
Apr 12 03:42:05 user nova-compute[72081]:   <name>instance-0000001f</name>
Apr 12 03:42:05 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:42:05 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:42:05 user nova-compute[72081]:   <metadata>
Apr 12 03:42:05 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:42:05 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:42:05 user nova-compute[72081]:       <nova:name>tempest-AttachVolumeTestJSON-server-2123178219</nova:name>
Apr 12 03:42:05 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:42:05</nova:creationTime>
Apr 12 03:42:05 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:42:05 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:42:05 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:42:05 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:42:05 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:42:05 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:42:05 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:42:05 user nova-compute[72081]:       <nova:owner>
Apr 12 03:42:05 user nova-compute[72081]:         <nova:user uuid="872fbc3d59274cd480c17b4b7342499a">tempest-AttachVolumeTestJSON-617614045-project-member</nova:user>
Apr 12 03:42:05 user nova-compute[72081]:         <nova:project uuid="a341e582192a47548df8627d82c96ea8">tempest-AttachVolumeTestJSON-617614045</nova:project>
Apr 12 03:42:05 user nova-compute[72081]:       </nova:owner>
Apr 12 03:42:05 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:42:05 user nova-compute[72081]:       <nova:ports>
Apr 12 03:42:05 user nova-compute[72081]:         <nova:port uuid="e16af111-f1d4-4f8c-a67a-c80f546c37b1">
Apr 12 03:42:05 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Apr 12 03:42:05 user nova-compute[72081]:         </nova:port>
Apr 12 03:42:05 user nova-compute[72081]:       </nova:ports>
Apr 12 03:42:05 user nova-compute[72081]:     </nova:instance>
Apr 12 03:42:05 user nova-compute[72081]:   </metadata>
Apr 12 03:42:05 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:42:05 user nova-compute[72081]:     <system>
Apr 12 03:42:05 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:42:05 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:42:05 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:42:05 user nova-compute[72081]:       <entry name="serial">606425bb-8968-41f9-b3a5-80b6fd188a67</entry>
Apr 12 03:42:05 user nova-compute[72081]:       <entry name="uuid">606425bb-8968-41f9-b3a5-80b6fd188a67</entry>
Apr 12 03:42:05 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:42:05 user nova-compute[72081]:     </system>
Apr 12 03:42:05 user nova-compute[72081]:   </sysinfo>
Apr 12 03:42:05 user nova-compute[72081]:   <os>
Apr 12 03:42:05 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:42:05 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:42:05 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:42:05 user nova-compute[72081]:   </os>
Apr 12 03:42:05 user nova-compute[72081]:   <features>
Apr 12 03:42:05 user nova-compute[72081]:     <acpi/>
Apr 12 03:42:05 user nova-compute[72081]:     <apic/>
Apr 12 03:42:05 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:42:05 user nova-compute[72081]:   </features>
Apr 12 03:42:05 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:42:05 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:42:05 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:42:05 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:42:05 user nova-compute[72081]:   </clock>
Apr 12 03:42:05 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:42:05 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:42:05 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:42:05 user nova-compute[72081]:   </cpu>
Apr 12 03:42:05 user nova-compute[72081]:   <devices>
Apr 12 03:42:05 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:42:05 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:42:05 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk"/>
Apr 12 03:42:05 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:42:05 user nova-compute[72081]:     </disk>
Apr 12 03:42:05 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:42:05 user nova-compute[72081]:       <mac address="fa:16:3e:85:88:9b"/>
Apr 12 03:42:05 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:42:05 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:42:05 user nova-compute[72081]:       <target dev="tape16af111-f1"/>
Apr 12 03:42:05 user nova-compute[72081]:     </interface>
Apr 12 03:42:05 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:42:05 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/console.log" append="off"/>
Apr 12 03:42:05 user nova-compute[72081]:     </serial>
Apr 12 03:42:05 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:42:05 user nova-compute[72081]:     <video>
Apr 12 03:42:05 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:42:05 user nova-compute[72081]:     </video>
Apr 12 03:42:05 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:42:05 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:42:05 user nova-compute[72081]:     </rng>
Apr 12 03:42:05 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:42:05 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:42:05 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:42:05 user nova-compute[72081]:     </memballoon>
Apr 12 03:42:05 user nova-compute[72081]:   </devices>
Apr 12 03:42:05 user nova-compute[72081]: </domain>
Apr 12 03:42:05 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:42:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-2123178219',display_name='tempest-AttachVolumeTestJSON-server-2123178219',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-2123178219',id=31,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDsSrynzjRvdumscunYM7qq2LINQa1dLMdBCKJ6XwNU+wH30BpZ+UxgkL98cPRjcfl2uMCQ+6nM9+XV2FMWC/Uf6P+0R1p0Yeexa689yM1BKJkKUWaai92yYlAVXCdH/1Q==',key_name='tempest-keypair-611227490',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a341e582192a47548df8627d82c96ea8',ramdisk_id='',reservation_id='r-qag42n01',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeTestJSON-617614045',owner_user_name='tempest-AttachVolumeTestJSON-617614045-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:42:03Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='872fbc3d59274cd480c17b4b7342499a',uuid=606425bb-8968-41f9-b3a5-80b6fd188a67,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Converting VIF {"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:85:88:9b,bridge_name='br-int',has_traffic_filtering=True,id=e16af111-f1d4-4f8c-a67a-c80f546c37b1,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16af111-f1') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG os_vif [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:85:88:9b,bridge_name='br-int',has_traffic_filtering=True,id=e16af111-f1d4-4f8c-a67a-c80f546c37b1,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16af111-f1') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tape16af111-f1, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tape16af111-f1, col_values=(('external_ids', {'iface-id': 'e16af111-f1d4-4f8c-a67a-c80f546c37b1', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:85:88:9b', 'vm-uuid': '606425bb-8968-41f9-b3a5-80b6fd188a67'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:05 user nova-compute[72081]: INFO os_vif [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:85:88:9b,bridge_name='br-int',has_traffic_filtering=True,id=e16af111-f1d4-4f8c-a67a-c80f546c37b1,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16af111-f1')
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] No VIF found with MAC fa:16:3e:85:88:9b, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Stashing vm_state: active {{(pid=72081) _prep_resize /opt/stack/nova/nova/compute/manager.py:5499}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.network.neutron [req-ae2a6615-d067-4aec-9d8c-803e68d98f9d req-f656e22c-5738-4e45-b154-3b8c4858f0f7 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Updated VIF entry in instance network info cache for port e16af111-f1d4-4f8c-a67a-c80f546c37b1. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.network.neutron [req-ae2a6615-d067-4aec-9d8c-803e68d98f9d req-f656e22c-5738-4e45-b154-3b8c4858f0f7 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Updating instance_info_cache with network_info: [{"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.resize_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.resize_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ae2a6615-d067-4aec-9d8c-803e68d98f9d req-f656e22c-5738-4e45-b154-3b8c4858f0f7 service nova] Releasing lock "refresh_cache-606425bb-8968-41f9-b3a5-80b6fd188a67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'pci_requests' on Instance uuid 48977eb2-ce43-4577-a1cb-340e35703dbd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:42:05 user nova-compute[72081]: INFO nova.compute.claims [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Claim successful on node user
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'resources' on Instance uuid 48977eb2-ce43-4577-a1cb-340e35703dbd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:42:05 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'pci_devices' on Instance uuid 48977eb2-ce43-4577-a1cb-340e35703dbd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:42:06 user nova-compute[72081]: INFO nova.compute.resource_tracker [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Updating resource usage from migration 35efd868-6d17-47a5-bc6f-dc21edd02c98
Apr 12 03:42:06 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:42:06 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:42:06 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:42:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.resize_claim" :: held 0.761s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:06 user nova-compute[72081]: INFO nova.compute.manager [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Migrating
Apr 12 03:42:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "compute-rpcapi-router" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:42:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquired lock "compute-rpcapi-router" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:42:06 user nova-compute[72081]: INFO nova.compute.rpcapi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Automatically selected compute RPC version 6.2 from minimum service version 66
Apr 12 03:42:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Releasing lock "compute-rpcapi-router" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:42:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:42:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquired lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:42:06 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:42:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:07 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Updating instance_info_cache with network_info: [{"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:42:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Releasing lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:42:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Starting migrate_disk_and_power_off {{(pid=72081) migrate_disk_and_power_off /opt/stack/nova/nova/virt/libvirt/driver.py:11460}}
Apr 12 03:42:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-30442059-c7b4-425d-a47f-319d3492d189 req-7012b597-4af4-4f65-936d-ab571b6395a0 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:42:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-30442059-c7b4-425d-a47f-319d3492d189 req-7012b597-4af4-4f65-936d-ab571b6395a0 service nova] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-30442059-c7b4-425d-a47f-319d3492d189 req-7012b597-4af4-4f65-936d-ab571b6395a0 service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-30442059-c7b4-425d-a47f-319d3492d189 req-7012b597-4af4-4f65-936d-ab571b6395a0 service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-30442059-c7b4-425d-a47f-319d3492d189 req-7012b597-4af4-4f65-936d-ab571b6395a0 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] No waiting events found dispatching network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:42:07 user nova-compute[72081]: WARNING nova.compute.manager [req-30442059-c7b4-425d-a47f-319d3492d189 req-7012b597-4af4-4f65-936d-ab571b6395a0 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received unexpected event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 for instance with vm_state building and task_state spawning.
Apr 12 03:42:07 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:42:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:07 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:42:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-8a3e649d-6306-41e3-91fa-f47b8e0005d0 req-cb1921e0-28c7-4201-94f7-c5d3515b00d7 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received event network-vif-unplugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:42:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8a3e649d-6306-41e3-91fa-f47b8e0005d0 req-cb1921e0-28c7-4201-94f7-c5d3515b00d7 service nova] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8a3e649d-6306-41e3-91fa-f47b8e0005d0 req-cb1921e0-28c7-4201-94f7-c5d3515b00d7 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8a3e649d-6306-41e3-91fa-f47b8e0005d0 req-cb1921e0-28c7-4201-94f7-c5d3515b00d7 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-8a3e649d-6306-41e3-91fa-f47b8e0005d0 req-cb1921e0-28c7-4201-94f7-c5d3515b00d7 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] No waiting events found dispatching network-vif-unplugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:42:07 user nova-compute[72081]: WARNING nova.compute.manager [req-8a3e649d-6306-41e3-91fa-f47b8e0005d0 req-cb1921e0-28c7-4201-94f7-c5d3515b00d7 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received unexpected event network-vif-unplugged-18f9f68c-eadd-4582-8f16-b88a13496852 for instance with vm_state active and task_state resize_migrating.
Apr 12 03:42:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:08 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Instance destroyed successfully.
Apr 12 03:42:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-12T03:33:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-2050314981',display_name='tempest-ServerActionsTestJSON-server-1847743592',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-2050314981',id=14,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKmrI9wy1CUIwhlKuXvzk0mD7Pt5nZFvI1CsqS0do6Lk76JQOvGl5lvx9rkz7UAM9WGocGXyWSOUOuKTtYZRD2HqKUbTN4VPOoliE1yuu+s0lx7vuZu45Mj7wDb2S8Ksyw==',key_name='tempest-keypair-364276420',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:37:25Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={rebuild='server'},migration_context=MigrationContext,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='815b65aa5ad9442db0a087c646f96bbd',ramdisk_id='',reservation_id='r-usz98205',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1898059859',owner_user_name='tempest-ServerActionsTestJSON-1898059859-project-member'},tags=<?>,task_state='resize_migrating',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:42:06Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a7f422f1a05a4a92ba387e424850b360',uuid=48977eb2-ce43-4577-a1cb-340e35703dbd,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1452412921-network", "vif_mac": "fa:16:3e:b3:c5:ca"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converting VIF {"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1452412921-network", "vif_mac": "fa:16:3e:b3:c5:ca"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG os_vif [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap18f9f68c-ea, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:08 user nova-compute[72081]: INFO os_vif [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea')
Apr 12 03:42:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YDhqrwbV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:42:08 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:42:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[25840485-9cc8-4494-bfe5-118372982a70]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:42:08 user nova-compute[72081]: No targets found.
Apr 12 03:42:08 user nova-compute[72081]: iSNS:
Apr 12 03:42:08 user nova-compute[72081]: No targets found.
Apr 12 03:42:08 user nova-compute[72081]: STATIC:
Apr 12 03:42:08 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf
Apr 12 03:42:08 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:42:08 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:42:08 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:42:08 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:42:08 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:42:08 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e
Apr 12 03:42:08 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:42:08 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:42:08 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742
Apr 12 03:42:08 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:42:08 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:42:08 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6
Apr 12 03:42:08 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:42:08 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:42:08 user nova-compute[72081]: FIRMWARE:
Apr 12 03:42:08 user nova-compute[72081]: No targets found.
Apr 12 03:42:08 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:42:08 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:42:08 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b3be87f8-840c-46c9-8377-60846a6153ac]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.035s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6a1ff0b9-9fae-42da-9ed9-0fea4dccfb20]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:42:08 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:42:08 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:42:08 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:42:08 user nova-compute[72081]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:42:08 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:42:08 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:42:08 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:42:08 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:42:08 user nova-compute[72081]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:42:08 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3f942d4310>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742'): ({'sdf'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Removing single pathed devices sdf {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a65c0df2-389b-43f6-996e-d10f607d1b6e]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdf {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdf" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ca69bb32-78fd-4bff-aa78-42ae0264068a]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Flushing IO for device /dev/sdf {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdf {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdf" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d53e6b9b-ccbe-4133-b713-fb5cda15d5bd]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Remove SCSI device /dev/sdf with /sys/block/sdf/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdf/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdf/device/delete" returned: 0 in 0.056s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[61aa176c-fa3e-4d42-8246-8a1acfc26b85]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Checking to see if SCSI volumes sdf have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] SCSI volumes sdf have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.033s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6e203f5f-b75c-47d0-896e-5f6173988cf7]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.171s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --logout" returned: 0 in 0.078s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[728858c1-61cb-4863-8114-d131bc80f593]: (4, ('Logging out of session [sid: 29, target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742, portal: 172.16.0.220,3260]\nLogout of [sid: 29, target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 29, target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742, portal: 172.16.0.220,3260]
Apr 12 03:42:08 user nova-compute[72081]: Logout of [sid: 29, target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742, portal: 172.16.0.220,3260] successful.
Apr 12 03:42:08 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.028s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[80e25fe4-a091-411b-9d82-ee260dbefb74]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.455s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] <== disconnect_volume: return (456ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdf (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] skipping disk /dev/sdf (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): cp -r /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd_resize/disk /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f88ecf99-d158-4f3c-8a87-6c0322fdc8ab/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f88ecf99-d158-4f3c-8a87-6c0322fdc8ab/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f88ecf99-d158-4f3c-8a87-6c0322fdc8ab/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f88ecf99-d158-4f3c-8a87-6c0322fdc8ab/disk --force-share --output=json" returned: 0 in 0.162s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/117ea473-f8e3-43c7-8406-d4590422cc0b/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CMD "cp -r /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd_resize/disk /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk" returned: 0 in 0.415s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): cp -r /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd_resize/disk.info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk.info {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CMD "cp -r /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd_resize/disk.info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk.info" returned: 0 in 0.039s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/117ea473-f8e3-43c7-8406-d4590422cc0b/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/117ea473-f8e3-43c7-8406-d4590422cc0b/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270929.6438663, 606425bb-8968-41f9-b3a5-80b6fd188a67 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:42:09 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] VM Resumed (Lifecycle Event)
Apr 12 03:42:09 user nova-compute[72081]: DEBUG nova.compute.manager [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:42:09 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Instance spawned successfully.
Apr 12 03:42:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-46f20efc-9329-445b-9f9b-d2ee7b6c7d63 req-f4ee2ece-5eb2-4b42-8562-f9fe7f212b28 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-46f20efc-9329-445b-9f9b-d2ee7b6c7d63 req-f4ee2ece-5eb2-4b42-8562-f9fe7f212b28 service nova] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-46f20efc-9329-445b-9f9b-d2ee7b6c7d63 req-f4ee2ece-5eb2-4b42-8562-f9fe7f212b28 service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-46f20efc-9329-445b-9f9b-d2ee7b6c7d63 req-f4ee2ece-5eb2-4b42-8562-f9fe7f212b28 service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-46f20efc-9329-445b-9f9b-d2ee7b6c7d63 req-f4ee2ece-5eb2-4b42-8562-f9fe7f212b28 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] No waiting events found dispatching network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:42:09 user nova-compute[72081]: WARNING nova.compute.manager [req-46f20efc-9329-445b-9f9b-d2ee7b6c7d63 req-f4ee2ece-5eb2-4b42-8562-f9fe7f212b28 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received unexpected event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 for instance with vm_state building and task_state spawning.
Apr 12 03:42:09 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:42:09 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:42:09 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270929.6453593, 606425bb-8968-41f9-b3a5-80b6fd188a67 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:42:09 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] VM Started (Lifecycle Event)
Apr 12 03:42:09 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/117ea473-f8e3-43c7-8406-d4590422cc0b/disk --force-share --output=json" returned: 0 in 0.181s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:09 user nova-compute[72081]: INFO nova.compute.manager [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Took 6.59 seconds to spawn the instance on the hypervisor.
Apr 12 03:42:09 user nova-compute[72081]: DEBUG nova.compute.manager [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:42:09 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:42:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-3c33b550-5ab7-4e15-a995-58f662760a94 req-3ab71e31-3822-438d-bbec-3730f8bae031 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3c33b550-5ab7-4e15-a995-58f662760a94 req-3ab71e31-3822-438d-bbec-3730f8bae031 service nova] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3c33b550-5ab7-4e15-a995-58f662760a94 req-3ab71e31-3822-438d-bbec-3730f8bae031 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3c33b550-5ab7-4e15-a995-58f662760a94 req-3ab71e31-3822-438d-bbec-3730f8bae031 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-3c33b550-5ab7-4e15-a995-58f662760a94 req-3ab71e31-3822-438d-bbec-3730f8bae031 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] No waiting events found dispatching network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:42:09 user nova-compute[72081]: WARNING nova.compute.manager [req-3c33b550-5ab7-4e15-a995-58f662760a94 req-3ab71e31-3822-438d-bbec-3730f8bae031 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received unexpected event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 for instance with vm_state active and task_state resize_migrating.
Apr 12 03:42:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.187s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:10 user nova-compute[72081]: INFO nova.compute.manager [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Took 7.72 seconds to build instance.
Apr 12 03:42:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-40783728-6a3d-4cff-b166-3d4f72f27cdc tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.834s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk --force-share --output=json" returned: 0 in 0.179s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:42:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk --force-share --output=json" returned: 0 in 0.159s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:10 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Port 18f9f68c-eadd-4582-8f16-b88a13496852 binding to destination host user is already ACTIVE {{(pid=72081) migrate_instance_start /opt/stack/nova/nova/network/neutron.py:3161}}
Apr 12 03:42:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636/disk --force-share --output=json" returned: 0 in 0.162s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdd (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:42:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:42:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:42:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:42:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquired lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:42:11 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:42:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.192s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:11 user nova-compute[72081]: DEBUG nova.objects.instance [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lazy-loading 'flavor' on Instance uuid 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:42:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.065s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.167s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:12 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:42:12 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:42:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=7641MB free_disk=25.960830688476562GB free_vcpus=1 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:12 user nova-compute[72081]: INFO nova.compute.manager [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Attaching volume 8568ebe9-126e-4516-87a9-cd53c4b520c3 to /dev/vdb
Apr 12 03:42:12 user nova-compute[72081]: DEBUG os_brick.utils [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3baddb37-9dd6-4de1-997d-a2e067f52ad3]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e5f64db-4279-4727-b461-dd2d1da9e4a8]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Applying migration context for instance 48977eb2-ce43-4577-a1cb-340e35703dbd as it has an incoming, in-progress migration 35efd868-6d17-47a5-bc6f-dc21edd02c98. Migration status is post-migrating {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:950}}
Apr 12 03:42:12 user nova-compute[72081]: INFO nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Updating resource usage from migration 35efd868-6d17-47a5-bc6f-dc21edd02c98
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4cf06999-dd9d-4c67-a8a9-1c4bff043e9f]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d943140c-7192-472e-bcf1-6d8bd1561e62]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 2d0904c3-74e0-47bb-a832-758a7035b2f5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance bcd267d2-5b2c-4ed2-b634-0210fae94dfd actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 1eef2c91-dfad-43aa-b193-25970db83b79 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance f73fa883-75f0-4593-a89c-d009c6d4ebf0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 3d4407f4-e078-4777-a110-cb3f922be2e8 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 117ea473-f8e3-43c7-8406-d4590422cc0b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 06481254-722a-4053-a6af-11f5dcc68636 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance f88ecf99-d158-4f3c-8a87-6c0322fdc8ab actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 614495f2-f108-4759-be60-bc6cc689c5b7 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 606425bb-8968-41f9-b3a5-80b6fd188a67 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Migration 35efd868-6d17-47a5-bc6f-dc21edd02c98 is active on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 128, 'DISK_GB': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1637}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 48977eb2-ce43-4577-a1cb-340e35703dbd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 192, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 13 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=2240MB phys_disk=40GB used_disk=10GB total_vcpus=12 used_vcpus=13 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Updating instance_info_cache with network_info: [{"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Releasing lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:42:12 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[adaa54b9-7ddc-46be-8d0d-208bddf3dc98]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG os_brick.utils [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] <== get_connector_properties: return (183ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Updating existing volume attachment record: 92dc0b3f-46f4-4637-ac0b-fe150f96bc7e {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG os_brick.utils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d82c032c-fc95-4bb0-9c04-f554f232307e]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8ed9ad8f-6680-49ce-a04e-48dc5efa00c9]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5e7f8875-f698-4e6d-9823-89c90939bc97]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ef143884-9e0c-4f9b-94e9-b67a6b468a00]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:42:12 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[faf4c0f4-ab4f-4464-81b3-87beb17ed910]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG os_brick.utils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] <== get_connector_properties: return (149ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:42:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.785s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:13 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:42:13 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:42:13 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:42:13 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:42:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-bcd267d2-5b2c-4ed2-b634-0210fae94dfd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:42:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-bcd267d2-5b2c-4ed2-b634-0210fae94dfd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:42:14 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 03:42:14 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Updating instance_info_cache with network_info: [{"id": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "address": "fa:16:3e:9d:0c:c2", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa4385731-05", "ovs_interfaceid": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:42:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-bcd267d2-5b2c-4ed2-b634-0210fae94dfd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:42:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 03:42:14 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:42:14 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:42:14 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:42:14 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:42:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'nDKzpQvs', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:42:17 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:42:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:42:17 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:42:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:42:17 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4bcdde4b-6344-40e7-a298-655deab58c71]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:42:17 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5e5dae2-fb4d-40cf-8c1c-445c9c21021a]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3] added
Apr 12 03:42:17 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2983faf8-2abd-4703-94b4-e4ea8dce84bd]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.028s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e8f397b8-f6e6-496d-9574-385c882dabdf]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v nDKzpQvs {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v nDKzpQvs" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6bb6e31d-bcf7-44bb-b50d-4cb07021f629]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'nDKzpQvs'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[be78d0c7-623b-41f2-9121-33c846b668b8]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Starting finish_migration {{(pid=72081) finish_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11647}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Instance directory exists: not creating {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4694}}
Apr 12 03:42:17 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Creating image(s)
Apr 12 03:42:17 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'trusted_certs' on Instance uuid 48977eb2-ce43-4577-a1cb-340e35703dbd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.038s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe5b75e3-59c2-4d4f-a7bf-4133dcbc64aa]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:42:17 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:42:17 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:42:17 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:42:17 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:42:17 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:42:17 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:42:17 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:42:17 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 -p 172.16.0.220:3260 --login" returned: 0 in 0.072s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4ce770db-fb4e-4b32-ba11-e1cad9362a6e]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3, portal: 172.16.0.220,3260]
Apr 12 03:42:17 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3, portal: 172.16.0.220,3260] successful.
Apr 12 03:42:17 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.146s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Checking if we can resize image /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[589b1808-3e72-4d66-b653-3c6d96c532ac]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3035bcbf-cf2e-4f3a-b72f-d7a6ebca0317]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:42:17 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:42:17 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:42:17 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:42:17 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:42:17 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:42:17 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:42:17 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:42:17 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:42:17 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:42:17 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.434s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] HCTL ('37', '-', '-', 0) found on session 30 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Scanning host 37 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8639bfca-e369-4d04-96d9-600135d7d7b3]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Searching for a device in session 30 and hctl ['37', '*', '*', 0] yield: sdf {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Connected to sdf using {'target_iqn': 'iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'nDKzpQvs', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.154s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Cannot resize image /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Did not create local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Ensure instance console log exists: /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Start _get_guest_xml network_info=[{"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1452412921-network", "vif_mac": "fa:16:3e:b3:c5:ca"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vdb', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YDhqrwbV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'attaching', 'instance': '48977eb2-ce43-4577-a1cb-340e35703dbd', 'attached_at': '2023-04-12T03:42:17.000000', 'detached_at': '', 'volume_id': '8278d82d-cc19-4660-b4f1-b13d92e7d742', 'serial': '8278d82d-cc19-4660-b4f1-b13d92e7d742'}, 'disk_bus': 'virtio', 'delete_on_termination': False, 'boot_index': None, 'guest_format': None, 'device_type': 'disk', 'attachment_id': 'af8956ba-b7d2-48e6-a127-576ac5ef71dd', 'volume_type': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:42:17 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:42:17 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:42:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:31Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='84',id=12,is_public=True,memory_mb=192,name='m1.micro',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'vcpu_model' on Instance uuid 48977eb2-ce43-4577-a1cb-340e35703dbd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:42:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YDhqrwbV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:42:17 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:42:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.506s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] <== connect_volume: return (1508ms) {'type': 'block', 'scsi_wwn': '26165626435326339', 'path': '/dev/sdf'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26165626435326339', 'path': '/dev/sdf'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.885s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:42:18 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:42:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:42:18 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG nova.objects.instance [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lazy-loading 'flavor' on Instance uuid 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1595c362-a769-4d29-83a6-bf0f641c573d]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:42:18 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] attach device xml: <disk type="block" device="disk">
Apr 12 03:42:18 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:42:18 user nova-compute[72081]:   <source dev="/dev/sdf"/>
Apr 12 03:42:18 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:42:18 user nova-compute[72081]:   <serial>8568ebe9-126e-4516-87a9-cd53c4b520c3</serial>
Apr 12 03:42:18 user nova-compute[72081]: </disk>
Apr 12 03:42:18 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3304f786-0ddc-48ef-b85c-b0a265ea035a]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742] added
Apr 12 03:42:18 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e0e9fadd-0740-4d82-9f83-7b31050bfb80]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c6037bb6-cdb2-46ff-9021-4c5242cdc4b7]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YDhqrwbV {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YDhqrwbV" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f7569c5e-a604-4012-91ae-c4f2fb76f2b8]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'YDhqrwbV'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6b9c77ea-bc43-464d-8bbe-93a92ba57a27]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[502ce758-eaa2-4b29-9d96-239e01aa8404]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:42:18 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:42:18 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:42:18 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:42:18 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:42:18 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:42:18 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:42:18 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:42:18 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:42:18 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:42:18 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --login" returned: 0 in 0.076s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b65245fd-8453-41a9-8791-6ec0b1047186]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742, portal: 172.16.0.220,3260]
Apr 12 03:42:18 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742, portal: 172.16.0.220,3260] successful.
Apr 12 03:42:18 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.038s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b5c5f81b-3648-4515-b271-275ad99a0584]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.034s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bbff6241-4fbe-4024-8ff8-83bddddc725e]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:42:18 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:42:18 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:42:18 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:42:18 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:42:18 user nova-compute[72081]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:42:18 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:42:18 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:42:18 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:42:18 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:42:18 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:42:18 user nova-compute[72081]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:42:18 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.339s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] HCTL ('38', '-', '-', 0) found on session 31 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Scanning host 38 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host38/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host38/scan" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[528f1caf-5be7-48fe-a7bc-bf128de26fbd]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:18 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Searching for a device in session 31 and hctl ['38', '*', '*', 0] yield: None {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] No VIF found with MAC fa:16:3e:84:c4:9d, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3abbfb3f-ffc6-4be8-b223-3f6f78823019 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.958s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Searching for a device in session 31 and hctl ['38', '*', '*', 0] yield: sdg {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Connected to sdg using {'target_iqn': 'iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YDhqrwbV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.370s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] <== connect_volume: return (2256ms) {'type': 'block', 'scsi_wwn': '26437356130623139', 'path': '/dev/sdg'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26437356130623139', 'path': '/dev/sdg'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-12T03:33:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-2050314981',display_name='tempest-ServerActionsTestJSON-server-1847743592',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(12),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-2050314981',id=14,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=12,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKmrI9wy1CUIwhlKuXvzk0mD7Pt5nZFvI1CsqS0do6Lk76JQOvGl5lvx9rkz7UAM9WGocGXyWSOUOuKTtYZRD2HqKUbTN4VPOoliE1yuu+s0lx7vuZu45Mj7wDb2S8Ksyw==',key_name='tempest-keypair-364276420',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:37:25Z,launched_on='user',locked=False,locked_by=None,memory_mb=192,metadata={rebuild='server'},migration_context=MigrationContext,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=Flavor(11),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='815b65aa5ad9442db0a087c646f96bbd',ramdisk_id='',reservation_id='r-usz98205',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1898059859',owner_user_name='tempest-ServerActionsTestJSON-1898059859-project-member'},tags=<?>,task_state='resize_finish',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:42:11Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a7f422f1a05a4a92ba387e424850b360',uuid=48977eb2-ce43-4577-a1cb-340e35703dbd,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1452412921-network", "vif_mac": "fa:16:3e:b3:c5:ca"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converting VIF {"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1452412921-network", "vif_mac": "fa:16:3e:b3:c5:ca"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:42:19 user nova-compute[72081]:   <uuid>48977eb2-ce43-4577-a1cb-340e35703dbd</uuid>
Apr 12 03:42:19 user nova-compute[72081]:   <name>instance-0000000e</name>
Apr 12 03:42:19 user nova-compute[72081]:   <memory>196608</memory>
Apr 12 03:42:19 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:42:19 user nova-compute[72081]:   <metadata>
Apr 12 03:42:19 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:42:19 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:42:19 user nova-compute[72081]:       <nova:name>tempest-ServerActionsTestJSON-server-1847743592</nova:name>
Apr 12 03:42:19 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:42:17</nova:creationTime>
Apr 12 03:42:19 user nova-compute[72081]:       <nova:flavor name="m1.micro">
Apr 12 03:42:19 user nova-compute[72081]:         <nova:memory>192</nova:memory>
Apr 12 03:42:19 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:42:19 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:42:19 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:42:19 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:42:19 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:42:19 user nova-compute[72081]:       <nova:owner>
Apr 12 03:42:19 user nova-compute[72081]:         <nova:user uuid="a7f422f1a05a4a92ba387e424850b360">tempest-ServerActionsTestJSON-1898059859-project-member</nova:user>
Apr 12 03:42:19 user nova-compute[72081]:         <nova:project uuid="815b65aa5ad9442db0a087c646f96bbd">tempest-ServerActionsTestJSON-1898059859</nova:project>
Apr 12 03:42:19 user nova-compute[72081]:       </nova:owner>
Apr 12 03:42:19 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:42:19 user nova-compute[72081]:       <nova:ports>
Apr 12 03:42:19 user nova-compute[72081]:         <nova:port uuid="18f9f68c-eadd-4582-8f16-b88a13496852">
Apr 12 03:42:19 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 12 03:42:19 user nova-compute[72081]:         </nova:port>
Apr 12 03:42:19 user nova-compute[72081]:       </nova:ports>
Apr 12 03:42:19 user nova-compute[72081]:     </nova:instance>
Apr 12 03:42:19 user nova-compute[72081]:   </metadata>
Apr 12 03:42:19 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:42:19 user nova-compute[72081]:     <system>
Apr 12 03:42:19 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:42:19 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:42:19 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:42:19 user nova-compute[72081]:       <entry name="serial">48977eb2-ce43-4577-a1cb-340e35703dbd</entry>
Apr 12 03:42:19 user nova-compute[72081]:       <entry name="uuid">48977eb2-ce43-4577-a1cb-340e35703dbd</entry>
Apr 12 03:42:19 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:42:19 user nova-compute[72081]:     </system>
Apr 12 03:42:19 user nova-compute[72081]:   </sysinfo>
Apr 12 03:42:19 user nova-compute[72081]:   <os>
Apr 12 03:42:19 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:42:19 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:42:19 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:42:19 user nova-compute[72081]:   </os>
Apr 12 03:42:19 user nova-compute[72081]:   <features>
Apr 12 03:42:19 user nova-compute[72081]:     <acpi/>
Apr 12 03:42:19 user nova-compute[72081]:     <apic/>
Apr 12 03:42:19 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:42:19 user nova-compute[72081]:   </features>
Apr 12 03:42:19 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:42:19 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:42:19 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:42:19 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:42:19 user nova-compute[72081]:   </clock>
Apr 12 03:42:19 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:42:19 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:42:19 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:42:19 user nova-compute[72081]:   </cpu>
Apr 12 03:42:19 user nova-compute[72081]:   <devices>
Apr 12 03:42:19 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:42:19 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:42:19 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk"/>
Apr 12 03:42:19 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:42:19 user nova-compute[72081]:     </disk>
Apr 12 03:42:19 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:42:19 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:42:19 user nova-compute[72081]:       <source dev="/dev/sdg"/>
Apr 12 03:42:19 user nova-compute[72081]:       <target dev="vdb" bus="virtio"/>
Apr 12 03:42:19 user nova-compute[72081]:       <serial>8278d82d-cc19-4660-b4f1-b13d92e7d742</serial>
Apr 12 03:42:19 user nova-compute[72081]:     </disk>
Apr 12 03:42:19 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:42:19 user nova-compute[72081]:       <mac address="fa:16:3e:b3:c5:ca"/>
Apr 12 03:42:19 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:42:19 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:42:19 user nova-compute[72081]:       <target dev="tap18f9f68c-ea"/>
Apr 12 03:42:19 user nova-compute[72081]:     </interface>
Apr 12 03:42:19 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:42:19 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/console.log" append="off"/>
Apr 12 03:42:19 user nova-compute[72081]:     </serial>
Apr 12 03:42:19 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:42:19 user nova-compute[72081]:     <video>
Apr 12 03:42:19 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:42:19 user nova-compute[72081]:     </video>
Apr 12 03:42:19 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:42:19 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:42:19 user nova-compute[72081]:     </rng>
Apr 12 03:42:19 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:42:19 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:42:19 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:42:19 user nova-compute[72081]:     </memballoon>
Apr 12 03:42:19 user nova-compute[72081]:   </devices>
Apr 12 03:42:19 user nova-compute[72081]: </domain>
Apr 12 03:42:19 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-12T03:33:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-2050314981',display_name='tempest-ServerActionsTestJSON-server-1847743592',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(12),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-2050314981',id=14,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=12,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKmrI9wy1CUIwhlKuXvzk0mD7Pt5nZFvI1CsqS0do6Lk76JQOvGl5lvx9rkz7UAM9WGocGXyWSOUOuKTtYZRD2HqKUbTN4VPOoliE1yuu+s0lx7vuZu45Mj7wDb2S8Ksyw==',key_name='tempest-keypair-364276420',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:37:25Z,launched_on='user',locked=False,locked_by=None,memory_mb=192,metadata={rebuild='server'},migration_context=MigrationContext,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=Flavor(11),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='815b65aa5ad9442db0a087c646f96bbd',ramdisk_id='',reservation_id='r-usz98205',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1898059859',owner_user_name='tempest-ServerActionsTestJSON-1898059859-project-member'},tags=<?>,task_state='resize_finish',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:42:11Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a7f422f1a05a4a92ba387e424850b360',uuid=48977eb2-ce43-4577-a1cb-340e35703dbd,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1452412921-network", "vif_mac": "fa:16:3e:b3:c5:ca"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converting VIF {"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1452412921-network", "vif_mac": "fa:16:3e:b3:c5:ca"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG os_vif [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap18f9f68c-ea, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap18f9f68c-ea, col_values=(('external_ids', {'iface-id': '18f9f68c-eadd-4582-8f16-b88a13496852', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:b3:c5:ca', 'vm-uuid': '48977eb2-ce43-4577-a1cb-340e35703dbd'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:19 user nova-compute[72081]: INFO os_vif [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea')
Apr 12 03:42:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:42:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] No VIF found with MAC fa:16:3e:b3:c5:ca, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:42:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:21 user nova-compute[72081]: DEBUG nova.compute.manager [req-f0037c84-4db8-4389-a6aa-286d05de0552 req-df347726-c339-4798-97d4-2235aa9fd51f service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:42:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f0037c84-4db8-4389-a6aa-286d05de0552 req-df347726-c339-4798-97d4-2235aa9fd51f service nova] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f0037c84-4db8-4389-a6aa-286d05de0552 req-df347726-c339-4798-97d4-2235aa9fd51f service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f0037c84-4db8-4389-a6aa-286d05de0552 req-df347726-c339-4798-97d4-2235aa9fd51f service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:21 user nova-compute[72081]: DEBUG nova.compute.manager [req-f0037c84-4db8-4389-a6aa-286d05de0552 req-df347726-c339-4798-97d4-2235aa9fd51f service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] No waiting events found dispatching network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:42:21 user nova-compute[72081]: WARNING nova.compute.manager [req-f0037c84-4db8-4389-a6aa-286d05de0552 req-df347726-c339-4798-97d4-2235aa9fd51f service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received unexpected event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 for instance with vm_state active and task_state resize_finish.
Apr 12 03:42:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:23 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681270928.2447312, 48977eb2-ce43-4577-a1cb-340e35703dbd => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:42:23 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] VM Stopped (Lifecycle Event)
Apr 12 03:42:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ee81d125-8d14-4df7-aa18-51cfaf17126c None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:42:23 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270943.7330782, 48977eb2-ce43-4577-a1cb-340e35703dbd => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:42:23 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] VM Resumed (Lifecycle Event)
Apr 12 03:42:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:42:23 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Instance running successfully.
Apr 12 03:42:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Failed to set time: agent not configured {{(pid=72081) sync_guest_time /opt/stack/nova/nova/virt/libvirt/guest.py:200}}
Apr 12 03:42:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9f05d6f1-9239-4381-880d-65d435739855 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] finish_migration finished successfully. {{(pid=72081) finish_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11742}}
Apr 12 03:42:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:42:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: resize_finish, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:42:23 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] During sync_power_state the instance has a pending task (resize_finish). Skip.
Apr 12 03:42:23 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270943.734825, 48977eb2-ce43-4577-a1cb-340e35703dbd => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:42:23 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] VM Started (Lifecycle Event)
Apr 12 03:42:23 user nova-compute[72081]: DEBUG nova.compute.manager [req-cc94be43-c143-44f9-a7c6-06a58516050f req-fb36c644-1320-4069-b701-f9ce0e930d5a service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Received event network-changed-c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:42:23 user nova-compute[72081]: DEBUG nova.compute.manager [req-cc94be43-c143-44f9-a7c6-06a58516050f req-fb36c644-1320-4069-b701-f9ce0e930d5a service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Refreshing instance network info cache due to event network-changed-c904583a-ea96-45c9-ad62-d5e0cc68031f. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:42:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc94be43-c143-44f9-a7c6-06a58516050f req-fb36c644-1320-4069-b701-f9ce0e930d5a service nova] Acquiring lock "refresh_cache-2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:42:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc94be43-c143-44f9-a7c6-06a58516050f req-fb36c644-1320-4069-b701-f9ce0e930d5a service nova] Acquired lock "refresh_cache-2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:42:23 user nova-compute[72081]: DEBUG nova.network.neutron [req-cc94be43-c143-44f9-a7c6-06a58516050f req-fb36c644-1320-4069-b701-f9ce0e930d5a service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Refreshing network info cache for port c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:42:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:42:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: resize_finish, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:42:23 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] During sync_power_state the instance has a pending task (resize_finish). Skip.
Apr 12 03:42:24 user nova-compute[72081]: DEBUG nova.compute.manager [req-b63a93d5-09e1-4c3b-8339-3e175a70fb7b req-40eae762-6108-4d4d-b386-1760df13ef84 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Received event network-changed-c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:42:24 user nova-compute[72081]: DEBUG nova.compute.manager [req-b63a93d5-09e1-4c3b-8339-3e175a70fb7b req-40eae762-6108-4d4d-b386-1760df13ef84 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Refreshing instance network info cache due to event network-changed-c904583a-ea96-45c9-ad62-d5e0cc68031f. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:42:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b63a93d5-09e1-4c3b-8339-3e175a70fb7b req-40eae762-6108-4d4d-b386-1760df13ef84 service nova] Acquiring lock "refresh_cache-2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:42:24 user nova-compute[72081]: DEBUG nova.network.neutron [req-cc94be43-c143-44f9-a7c6-06a58516050f req-fb36c644-1320-4069-b701-f9ce0e930d5a service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Updated VIF entry in instance network info cache for port c904583a-ea96-45c9-ad62-d5e0cc68031f. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:42:24 user nova-compute[72081]: DEBUG nova.network.neutron [req-cc94be43-c143-44f9-a7c6-06a58516050f req-fb36c644-1320-4069-b701-f9ce0e930d5a service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Updating instance_info_cache with network_info: [{"id": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "address": "fa:16:3e:84:c4:9d", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.10", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc904583a-ea", "ovs_interfaceid": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:42:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc94be43-c143-44f9-a7c6-06a58516050f req-fb36c644-1320-4069-b701-f9ce0e930d5a service nova] Releasing lock "refresh_cache-2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:42:24 user nova-compute[72081]: DEBUG nova.compute.manager [req-cc94be43-c143-44f9-a7c6-06a58516050f req-fb36c644-1320-4069-b701-f9ce0e930d5a service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:42:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc94be43-c143-44f9-a7c6-06a58516050f req-fb36c644-1320-4069-b701-f9ce0e930d5a service nova] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc94be43-c143-44f9-a7c6-06a58516050f req-fb36c644-1320-4069-b701-f9ce0e930d5a service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cc94be43-c143-44f9-a7c6-06a58516050f req-fb36c644-1320-4069-b701-f9ce0e930d5a service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:24 user nova-compute[72081]: DEBUG nova.compute.manager [req-cc94be43-c143-44f9-a7c6-06a58516050f req-fb36c644-1320-4069-b701-f9ce0e930d5a service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] No waiting events found dispatching network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:42:24 user nova-compute[72081]: WARNING nova.compute.manager [req-cc94be43-c143-44f9-a7c6-06a58516050f req-fb36c644-1320-4069-b701-f9ce0e930d5a service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received unexpected event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 for instance with vm_state active and task_state resize_finish.
Apr 12 03:42:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b63a93d5-09e1-4c3b-8339-3e175a70fb7b req-40eae762-6108-4d4d-b386-1760df13ef84 service nova] Acquired lock "refresh_cache-2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:42:24 user nova-compute[72081]: DEBUG nova.network.neutron [req-b63a93d5-09e1-4c3b-8339-3e175a70fb7b req-40eae762-6108-4d4d-b386-1760df13ef84 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Refreshing network info cache for port c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:42:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c54adbd6-a396-4cb6-a0d5-cd6fe0bdef71 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c54adbd6-a396-4cb6-a0d5-cd6fe0bdef71 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" acquired by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:24 user nova-compute[72081]: INFO nova.compute.manager [None req-c54adbd6-a396-4cb6-a0d5-cd6fe0bdef71 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Rebooting instance
Apr 12 03:42:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c54adbd6-a396-4cb6-a0d5-cd6fe0bdef71 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "refresh_cache-2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:42:25 user nova-compute[72081]: DEBUG nova.network.neutron [req-b63a93d5-09e1-4c3b-8339-3e175a70fb7b req-40eae762-6108-4d4d-b386-1760df13ef84 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Updated VIF entry in instance network info cache for port c904583a-ea96-45c9-ad62-d5e0cc68031f. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:42:25 user nova-compute[72081]: DEBUG nova.network.neutron [req-b63a93d5-09e1-4c3b-8339-3e175a70fb7b req-40eae762-6108-4d4d-b386-1760df13ef84 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Updating instance_info_cache with network_info: [{"id": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "address": "fa:16:3e:84:c4:9d", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.10", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc904583a-ea", "ovs_interfaceid": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:42:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b63a93d5-09e1-4c3b-8339-3e175a70fb7b req-40eae762-6108-4d4d-b386-1760df13ef84 service nova] Releasing lock "refresh_cache-2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:42:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c54adbd6-a396-4cb6-a0d5-cd6fe0bdef71 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquired lock "refresh_cache-2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:42:25 user nova-compute[72081]: DEBUG nova.network.neutron [None req-c54adbd6-a396-4cb6-a0d5-cd6fe0bdef71 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:42:26 user nova-compute[72081]: DEBUG nova.network.neutron [None req-c54adbd6-a396-4cb6-a0d5-cd6fe0bdef71 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Updating instance_info_cache with network_info: [{"id": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "address": "fa:16:3e:84:c4:9d", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.10", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc904583a-ea", "ovs_interfaceid": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:42:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c54adbd6-a396-4cb6-a0d5-cd6fe0bdef71 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Releasing lock "refresh_cache-2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:42:26 user nova-compute[72081]: DEBUG nova.compute.manager [None req-c54adbd6-a396-4cb6-a0d5-cd6fe0bdef71 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:42:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-eb1366d1-8e9f-485e-bb13-1fca8cab2ab2 req-9327de2c-360a-403e-9999-fec6a03dbdcc service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Received event network-vif-unplugged-c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:42:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-eb1366d1-8e9f-485e-bb13-1fca8cab2ab2 req-9327de2c-360a-403e-9999-fec6a03dbdcc service nova] Acquiring lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-eb1366d1-8e9f-485e-bb13-1fca8cab2ab2 req-9327de2c-360a-403e-9999-fec6a03dbdcc service nova] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-eb1366d1-8e9f-485e-bb13-1fca8cab2ab2 req-9327de2c-360a-403e-9999-fec6a03dbdcc service nova] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-eb1366d1-8e9f-485e-bb13-1fca8cab2ab2 req-9327de2c-360a-403e-9999-fec6a03dbdcc service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] No waiting events found dispatching network-vif-unplugged-c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:42:28 user nova-compute[72081]: WARNING nova.compute.manager [req-eb1366d1-8e9f-485e-bb13-1fca8cab2ab2 req-9327de2c-360a-403e-9999-fec6a03dbdcc service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Received unexpected event network-vif-unplugged-c904583a-ea96-45c9-ad62-d5e0cc68031f for instance with vm_state active and task_state reboot_started.
Apr 12 03:42:29 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-c54adbd6-a396-4cb6-a0d5-cd6fe0bdef71 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Instance shutdown successfully.
Apr 12 03:42:29 user nova-compute[72081]: DEBUG nova.compute.manager [req-ccd95568-d458-4ddb-9731-c916aeb658f7 req-f207340a-bcba-4921-b6ee-551f42e6c98b service nova] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Received event network-changed-ec1c8d01-1c15-4a89-9205-23f04ec6fc58 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:42:29 user nova-compute[72081]: DEBUG nova.compute.manager [req-ccd95568-d458-4ddb-9731-c916aeb658f7 req-f207340a-bcba-4921-b6ee-551f42e6c98b service nova] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Refreshing instance network info cache due to event network-changed-ec1c8d01-1c15-4a89-9205-23f04ec6fc58. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:42:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ccd95568-d458-4ddb-9731-c916aeb658f7 req-f207340a-bcba-4921-b6ee-551f42e6c98b service nova] Acquiring lock "refresh_cache-117ea473-f8e3-43c7-8406-d4590422cc0b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:42:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ccd95568-d458-4ddb-9731-c916aeb658f7 req-f207340a-bcba-4921-b6ee-551f42e6c98b service nova] Acquired lock "refresh_cache-117ea473-f8e3-43c7-8406-d4590422cc0b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:42:29 user nova-compute[72081]: DEBUG nova.network.neutron [req-ccd95568-d458-4ddb-9731-c916aeb658f7 req-f207340a-bcba-4921-b6ee-551f42e6c98b service nova] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Refreshing network info cache for port ec1c8d01-1c15-4a89-9205-23f04ec6fc58 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:42:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:30 user nova-compute[72081]: DEBUG nova.network.neutron [req-ccd95568-d458-4ddb-9731-c916aeb658f7 req-f207340a-bcba-4921-b6ee-551f42e6c98b service nova] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Updated VIF entry in instance network info cache for port ec1c8d01-1c15-4a89-9205-23f04ec6fc58. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:42:30 user nova-compute[72081]: DEBUG nova.network.neutron [req-ccd95568-d458-4ddb-9731-c916aeb658f7 req-f207340a-bcba-4921-b6ee-551f42e6c98b service nova] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Updating instance_info_cache with network_info: [{"id": "ec1c8d01-1c15-4a89-9205-23f04ec6fc58", "address": "fa:16:3e:3a:99:8b", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.20", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapec1c8d01-1c", "ovs_interfaceid": "ec1c8d01-1c15-4a89-9205-23f04ec6fc58", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:42:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ccd95568-d458-4ddb-9731-c916aeb658f7 req-f207340a-bcba-4921-b6ee-551f42e6c98b service nova] Releasing lock "refresh_cache-117ea473-f8e3-43c7-8406-d4590422cc0b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:42:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "1eef2c91-dfad-43aa-b193-25970db83b79" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "1eef2c91-dfad-43aa-b193-25970db83b79" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "1eef2c91-dfad-43aa-b193-25970db83b79-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "1eef2c91-dfad-43aa-b193-25970db83b79-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "1eef2c91-dfad-43aa-b193-25970db83b79-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:30 user nova-compute[72081]: INFO nova.compute.manager [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Terminating instance
Apr 12 03:42:30 user nova-compute[72081]: DEBUG nova.compute.manager [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:42:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG nova.compute.manager [req-a78207d1-f1dd-451d-836a-4bf1b22178b7 req-e308d034-a68d-45d3-a525-79412bca0bf7 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Received event network-vif-plugged-c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a78207d1-f1dd-451d-836a-4bf1b22178b7 req-e308d034-a68d-45d3-a525-79412bca0bf7 service nova] Acquiring lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a78207d1-f1dd-451d-836a-4bf1b22178b7 req-e308d034-a68d-45d3-a525-79412bca0bf7 service nova] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a78207d1-f1dd-451d-836a-4bf1b22178b7 req-e308d034-a68d-45d3-a525-79412bca0bf7 service nova] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG nova.compute.manager [req-a78207d1-f1dd-451d-836a-4bf1b22178b7 req-e308d034-a68d-45d3-a525-79412bca0bf7 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] No waiting events found dispatching network-vif-plugged-c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:42:31 user nova-compute[72081]: WARNING nova.compute.manager [req-a78207d1-f1dd-451d-836a-4bf1b22178b7 req-e308d034-a68d-45d3-a525-79412bca0bf7 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Received unexpected event network-vif-plugged-c904583a-ea96-45c9-ad62-d5e0cc68031f for instance with vm_state active and task_state reboot_started.
Apr 12 03:42:31 user nova-compute[72081]: DEBUG nova.compute.manager [req-a78207d1-f1dd-451d-836a-4bf1b22178b7 req-e308d034-a68d-45d3-a525-79412bca0bf7 service nova] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Received event network-vif-unplugged-50b78ba6-f017-4643-a2ec-fa0f348bb270 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a78207d1-f1dd-451d-836a-4bf1b22178b7 req-e308d034-a68d-45d3-a525-79412bca0bf7 service nova] Acquiring lock "1eef2c91-dfad-43aa-b193-25970db83b79-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a78207d1-f1dd-451d-836a-4bf1b22178b7 req-e308d034-a68d-45d3-a525-79412bca0bf7 service nova] Lock "1eef2c91-dfad-43aa-b193-25970db83b79-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a78207d1-f1dd-451d-836a-4bf1b22178b7 req-e308d034-a68d-45d3-a525-79412bca0bf7 service nova] Lock "1eef2c91-dfad-43aa-b193-25970db83b79-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG nova.compute.manager [req-a78207d1-f1dd-451d-836a-4bf1b22178b7 req-e308d034-a68d-45d3-a525-79412bca0bf7 service nova] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] No waiting events found dispatching network-vif-unplugged-50b78ba6-f017-4643-a2ec-fa0f348bb270 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG nova.compute.manager [req-a78207d1-f1dd-451d-836a-4bf1b22178b7 req-e308d034-a68d-45d3-a525-79412bca0bf7 service nova] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Received event network-vif-unplugged-50b78ba6-f017-4643-a2ec-fa0f348bb270 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-431a073c-ab43-4d1d-bece-e405cdd0564b tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "117ea473-f8e3-43c7-8406-d4590422cc0b" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-431a073c-ab43-4d1d-bece-e405cdd0564b tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "117ea473-f8e3-43c7-8406-d4590422cc0b" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-431a073c-ab43-4d1d-bece-e405cdd0564b tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "117ea473-f8e3-43c7-8406-d4590422cc0b-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-431a073c-ab43-4d1d-bece-e405cdd0564b tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "117ea473-f8e3-43c7-8406-d4590422cc0b-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-431a073c-ab43-4d1d-bece-e405cdd0564b tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "117ea473-f8e3-43c7-8406-d4590422cc0b-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:31 user nova-compute[72081]: INFO nova.compute.manager [None req-431a073c-ab43-4d1d-bece-e405cdd0564b tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Terminating instance
Apr 12 03:42:31 user nova-compute[72081]: DEBUG nova.compute.manager [None req-431a073c-ab43-4d1d-bece-e405cdd0564b tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:31 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Instance destroyed successfully.
Apr 12 03:42:31 user nova-compute[72081]: DEBUG nova.objects.instance [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lazy-loading 'resources' on Instance uuid 1eef2c91-dfad-43aa-b193-25970db83b79 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:37:52Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-volume-backed-server-966703655',display_name='tempest-TestVolumeBootPattern-volume-backed-server-966703655',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-volume-backed-server-966703655',id=21,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:38:03Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='003688353e2f4b36831c574ebaf65173',ramdisk_id='',reservation_id='r-ux4uytew',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeBootPattern-235043461',owner_user_name='tempest-TestVolumeBootPattern-235043461-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:38:04Z,user_data=None,user_id='b00b348933224558aad1cb12921f100f',uuid=1eef2c91-dfad-43aa-b193-25970db83b79,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "50b78ba6-f017-4643-a2ec-fa0f348bb270", "address": "fa:16:3e:60:70:2a", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50b78ba6-f0", "ovs_interfaceid": "50b78ba6-f017-4643-a2ec-fa0f348bb270", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converting VIF {"id": "50b78ba6-f017-4643-a2ec-fa0f348bb270", "address": "fa:16:3e:60:70:2a", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50b78ba6-f0", "ovs_interfaceid": "50b78ba6-f017-4643-a2ec-fa0f348bb270", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:60:70:2a,bridge_name='br-int',has_traffic_filtering=True,id=50b78ba6-f017-4643-a2ec-fa0f348bb270,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap50b78ba6-f0') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG os_vif [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:60:70:2a,bridge_name='br-int',has_traffic_filtering=True,id=50b78ba6-f017-4643-a2ec-fa0f348bb270,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap50b78ba6-f0') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap50b78ba6-f0, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:31 user nova-compute[72081]: INFO os_vif [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:60:70:2a,bridge_name='br-int',has_traffic_filtering=True,id=50b78ba6-f017-4643-a2ec-fa0f348bb270,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap50b78ba6-f0')
Apr 12 03:42:31 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'zxOqldvX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:42:31 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:42:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[653cdc1e-69c4-4146-ad44-4f38aafc03b1]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:42:31 user nova-compute[72081]: No targets found.
Apr 12 03:42:31 user nova-compute[72081]: iSNS:
Apr 12 03:42:31 user nova-compute[72081]: No targets found.
Apr 12 03:42:31 user nova-compute[72081]: STATIC:
Apr 12 03:42:31 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf
Apr 12 03:42:31 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:42:31 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:42:31 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:42:31 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:42:31 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:42:31 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e
Apr 12 03:42:31 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:42:31 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:42:31 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3
Apr 12 03:42:31 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:42:31 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:42:31 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742
Apr 12 03:42:31 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:42:31 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:42:31 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6
Apr 12 03:42:31 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:42:31 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:42:31 user nova-compute[72081]: FIRMWARE:
Apr 12 03:42:31 user nova-compute[72081]: No targets found.
Apr 12 03:42:31 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:42:31 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:42:31 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[af2e9d7c-6ad5-4463-9599-a2c58b904d72]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.031s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6762ad8f-5507-4957-92bf-7f8f7c18c548]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:42:31 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:42:31 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:42:31 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:42:31 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:42:31 user nova-compute[72081]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:42:31 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:42:31 user nova-compute[72081]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf (non-flash)
Apr 12 03:42:31 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:42:31 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:42:31 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:42:31 user nova-compute[72081]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:42:31 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fac137a30>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf'): ({'sde'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Removing single pathed devices sde {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b6299a5a-0fdc-44be-97bc-060ebab51da7]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b9b5c9c5-3e91-4952-8655-dd21898929c0]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Flushing IO for device /dev/sde {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[806aa092-38d1-4841-a88e-6ed670e7121a]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.048s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[16f2bbac-d9aa-413c-bf81-651a670896fe]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] SCSI volumes sde have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[73767c53-e596-4652-965a-9a2490832fd8]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:31 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Instance destroyed successfully.
Apr 12 03:42:31 user nova-compute[72081]: DEBUG nova.objects.instance [None req-431a073c-ab43-4d1d-bece-e405cdd0564b tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lazy-loading 'resources' on Instance uuid 117ea473-f8e3-43c7-8406-d4590422cc0b {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf -p 172.16.0.220:3260 --logout" returned: 0 in 0.099s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[fc188875-f7a5-4856-811f-cda7a6d8600b]: (4, ('Logging out of session [sid: 25, target: iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf, portal: 172.16.0.220,3260]\nLogout of [sid: 25, target: iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 25, target: iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf, portal: 172.16.0.220,3260]
Apr 12 03:42:31 user nova-compute[72081]: Logout of [sid: 25, target: iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf, portal: 172.16.0.220,3260] successful.
Apr 12 03:42:31 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-431a073c-ab43-4d1d-bece-e405cdd0564b tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:40:38Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-1563626038',display_name='tempest-AttachVolumeNegativeTest-server-1563626038',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-1563626038',id=28,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFyKNKT4wgJznr+Ex2kNQaN9dpRqnWsyzDU9MRy0fmokHPPmcvE5+ezeiXGgA2zAoziwbaT10GVLc8cJoTbEoKhsJqM+5rbndvelFILpQi0tIFZnPn0XCK6f6SU5ZH8k7g==',key_name='tempest-keypair-819201420',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:40:46Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='d13546149a4e44e1b782b99e9eb452e8',ramdisk_id='',reservation_id='r-nw8w4ad9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeNegativeTest-494105310',owner_user_name='tempest-AttachVolumeNegativeTest-494105310-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:40:46Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d52637cca2114e5e99e688ee371c835a',uuid=117ea473-f8e3-43c7-8406-d4590422cc0b,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "ec1c8d01-1c15-4a89-9205-23f04ec6fc58", "address": "fa:16:3e:3a:99:8b", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.20", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapec1c8d01-1c", "ovs_interfaceid": "ec1c8d01-1c15-4a89-9205-23f04ec6fc58", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-431a073c-ab43-4d1d-bece-e405cdd0564b tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Converting VIF {"id": "ec1c8d01-1c15-4a89-9205-23f04ec6fc58", "address": "fa:16:3e:3a:99:8b", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.20", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapec1c8d01-1c", "ovs_interfaceid": "ec1c8d01-1c15-4a89-9205-23f04ec6fc58", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-431a073c-ab43-4d1d-bece-e405cdd0564b tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:3a:99:8b,bridge_name='br-int',has_traffic_filtering=True,id=ec1c8d01-1c15-4a89-9205-23f04ec6fc58,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapec1c8d01-1c') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG os_vif [None req-431a073c-ab43-4d1d-bece-e405cdd0564b tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:3a:99:8b,bridge_name='br-int',has_traffic_filtering=True,id=ec1c8d01-1c15-4a89-9205-23f04ec6fc58,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapec1c8d01-1c') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapec1c8d01-1c, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf -p 172.16.0.220:3260 --op delete" returned: 0 in 0.032s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[04897e44-2bc8-46d5-b0fc-b30435ba428d]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.394s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] <== disconnect_volume: return (395ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:42:31 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:42:31 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Deleting instance files /opt/stack/data/nova/instances/1eef2c91-dfad-43aa-b193-25970db83b79_del
Apr 12 03:42:31 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Deletion of /opt/stack/data/nova/instances/1eef2c91-dfad-43aa-b193-25970db83b79_del complete
Apr 12 03:42:31 user nova-compute[72081]: INFO os_vif [None req-431a073c-ab43-4d1d-bece-e405cdd0564b tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:3a:99:8b,bridge_name='br-int',has_traffic_filtering=True,id=ec1c8d01-1c15-4a89-9205-23f04ec6fc58,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapec1c8d01-1c')
Apr 12 03:42:31 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-431a073c-ab43-4d1d-bece-e405cdd0564b tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Deleting instance files /opt/stack/data/nova/instances/117ea473-f8e3-43c7-8406-d4590422cc0b_del
Apr 12 03:42:31 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-431a073c-ab43-4d1d-bece-e405cdd0564b tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Deletion of /opt/stack/data/nova/instances/117ea473-f8e3-43c7-8406-d4590422cc0b_del complete
Apr 12 03:42:32 user nova-compute[72081]: INFO nova.compute.manager [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Took 1.50 seconds to destroy the instance on the hypervisor.
Apr 12 03:42:32 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:42:32 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:42:32 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:42:32 user nova-compute[72081]: INFO nova.compute.manager [None req-431a073c-ab43-4d1d-bece-e405cdd0564b tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Took 0.78 seconds to destroy the instance on the hypervisor.
Apr 12 03:42:32 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-431a073c-ab43-4d1d-bece-e405cdd0564b tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:42:32 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:42:32 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:42:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:42:33 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Took 1.02 seconds to deallocate network for instance.
Apr 12 03:42:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Received event network-vif-plugged-50b78ba6-f017-4643-a2ec-fa0f348bb270 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] Acquiring lock "1eef2c91-dfad-43aa-b193-25970db83b79-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] Lock "1eef2c91-dfad-43aa-b193-25970db83b79-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] Lock "1eef2c91-dfad-43aa-b193-25970db83b79-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] No waiting events found dispatching network-vif-plugged-50b78ba6-f017-4643-a2ec-fa0f348bb270 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:42:33 user nova-compute[72081]: WARNING nova.compute.manager [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Received unexpected event network-vif-plugged-50b78ba6-f017-4643-a2ec-fa0f348bb270 for instance with vm_state active and task_state deleting.
Apr 12 03:42:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Received event network-vif-plugged-c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] Acquiring lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] No waiting events found dispatching network-vif-plugged-c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:42:33 user nova-compute[72081]: WARNING nova.compute.manager [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Received unexpected event network-vif-plugged-c904583a-ea96-45c9-ad62-d5e0cc68031f for instance with vm_state active and task_state reboot_started.
Apr 12 03:42:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Received event network-vif-plugged-c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] Acquiring lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] No waiting events found dispatching network-vif-plugged-c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:42:33 user nova-compute[72081]: WARNING nova.compute.manager [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Received unexpected event network-vif-plugged-c904583a-ea96-45c9-ad62-d5e0cc68031f for instance with vm_state active and task_state reboot_started.
Apr 12 03:42:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Received event network-vif-unplugged-ec1c8d01-1c15-4a89-9205-23f04ec6fc58 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] Acquiring lock "117ea473-f8e3-43c7-8406-d4590422cc0b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] Lock "117ea473-f8e3-43c7-8406-d4590422cc0b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] Lock "117ea473-f8e3-43c7-8406-d4590422cc0b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] No waiting events found dispatching network-vif-unplugged-ec1c8d01-1c15-4a89-9205-23f04ec6fc58 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Received event network-vif-unplugged-ec1c8d01-1c15-4a89-9205-23f04ec6fc58 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Received event network-vif-plugged-ec1c8d01-1c15-4a89-9205-23f04ec6fc58 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] Acquiring lock "117ea473-f8e3-43c7-8406-d4590422cc0b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] Lock "117ea473-f8e3-43c7-8406-d4590422cc0b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] Lock "117ea473-f8e3-43c7-8406-d4590422cc0b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] No waiting events found dispatching network-vif-plugged-ec1c8d01-1c15-4a89-9205-23f04ec6fc58 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:42:33 user nova-compute[72081]: WARNING nova.compute.manager [req-f931700c-6e44-4927-9711-a9ab2a77a960 req-922127b7-4348-4638-9be8-90f672746d88 service nova] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Received unexpected event network-vif-plugged-ec1c8d01-1c15-4a89-9205-23f04ec6fc58 for instance with vm_state active and task_state deleting.
Apr 12 03:42:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:42:33 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Took 1.19 seconds to deallocate network for instance.
Apr 12 03:42:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-431a073c-ab43-4d1d-bece-e405cdd0564b tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-431a073c-ab43-4d1d-bece-e405cdd0564b tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Removed pending event for 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170 due to event {{(pid=72081) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270953.5216882, 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:42:33 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] VM Resumed (Lifecycle Event)
Apr 12 03:42:33 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Instance running successfully.
Apr 12 03:42:33 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-c54adbd6-a396-4cb6-a0d5-cd6fe0bdef71 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Instance soft rebooted successfully.
Apr 12 03:42:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-c54adbd6-a396-4cb6-a0d5-cd6fe0bdef71 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: reboot_started, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:42:33 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] During sync_power_state the instance has a pending task (reboot_started). Skip.
Apr 12 03:42:33 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681270953.52294, 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:42:33 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] VM Started (Lifecycle Event)
Apr 12 03:42:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c54adbd6-a396-4cb6-a0d5-cd6fe0bdef71 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" "released" by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" :: held 8.632s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-744b6933-2c1f-4a39-9220-0395672bde40 req-578e380e-b961-467d-8af1-3fcf1403ac30 service nova] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Received event network-vif-deleted-50b78ba6-f017-4643-a2ec-fa0f348bb270 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-431a073c-ab43-4d1d-bece-e405cdd0564b tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-431a073c-ab43-4d1d-bece-e405cdd0564b tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-431a073c-ab43-4d1d-bece-e405cdd0564b tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.496s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:33 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-431a073c-ab43-4d1d-bece-e405cdd0564b tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Deleted allocations for instance 117ea473-f8e3-43c7-8406-d4590422cc0b
Apr 12 03:42:33 user nova-compute[72081]: INFO nova.compute.manager [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Took 0.91 seconds to detach 1 volumes for instance.
Apr 12 03:42:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Deleting volume: fa33fb38-f437-49cf-b0b6-b5f9e9cbb8cf {{(pid=72081) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3202}}
Apr 12 03:42:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-431a073c-ab43-4d1d-bece-e405cdd0564b tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "117ea473-f8e3-43c7-8406-d4590422cc0b" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.702s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:42:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:42:34 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:42:34 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:42:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.489s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:34 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Deleted allocations for instance 1eef2c91-dfad-43aa-b193-25970db83b79
Apr 12 03:42:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2f8b05fe-d3e6-4048-aacb-5cf6ebca19c2 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "1eef2c91-dfad-43aa-b193-25970db83b79" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 4.196s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:42:35 user nova-compute[72081]: DEBUG nova.compute.manager [req-8c10da47-71cd-47bf-a385-e2f50584a437 req-2d334c7f-3d4a-40ca-b370-da840a4db611 service nova] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Received event network-vif-deleted-ec1c8d01-1c15-4a89-9205-23f04ec6fc58 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:42:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:46 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681270951.4590068, 1eef2c91-dfad-43aa-b193-25970db83b79 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:42:46 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] VM Stopped (Lifecycle Event)
Apr 12 03:42:46 user nova-compute[72081]: DEBUG nova.compute.manager [None req-231298b3-0ac8-4b92-a037-8043ccd4d3b2 None None] [instance: 1eef2c91-dfad-43aa-b193-25970db83b79] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:42:46 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681270951.8923602, 117ea473-f8e3-43c7-8406-d4590422cc0b => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:42:46 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] VM Stopped (Lifecycle Event)
Apr 12 03:42:46 user nova-compute[72081]: DEBUG nova.compute.manager [None req-013c0309-25c1-406c-a02a-f6b738b92f9e None None] [instance: 117ea473-f8e3-43c7-8406-d4590422cc0b] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:42:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:42:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:05 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:43:06 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-cd85aefe-22ee-42f6-8afe-be96d01ae05f tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "06481254-722a-4053-a6af-11f5dcc68636" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-cd85aefe-22ee-42f6-8afe-be96d01ae05f tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "06481254-722a-4053-a6af-11f5dcc68636" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-cd85aefe-22ee-42f6-8afe-be96d01ae05f tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "06481254-722a-4053-a6af-11f5dcc68636-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-cd85aefe-22ee-42f6-8afe-be96d01ae05f tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "06481254-722a-4053-a6af-11f5dcc68636-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-cd85aefe-22ee-42f6-8afe-be96d01ae05f tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "06481254-722a-4053-a6af-11f5dcc68636-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:08 user nova-compute[72081]: INFO nova.compute.manager [None req-cd85aefe-22ee-42f6-8afe-be96d01ae05f tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Terminating instance
Apr 12 03:43:08 user nova-compute[72081]: DEBUG nova.compute.manager [None req-cd85aefe-22ee-42f6-8afe-be96d01ae05f tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG nova.compute.manager [req-a7df4609-d687-4773-b5b3-a8184d925b6b req-70c5a9ce-d503-4485-993e-42a595cb6c60 service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Received event network-vif-unplugged-5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a7df4609-d687-4773-b5b3-a8184d925b6b req-70c5a9ce-d503-4485-993e-42a595cb6c60 service nova] Acquiring lock "06481254-722a-4053-a6af-11f5dcc68636-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a7df4609-d687-4773-b5b3-a8184d925b6b req-70c5a9ce-d503-4485-993e-42a595cb6c60 service nova] Lock "06481254-722a-4053-a6af-11f5dcc68636-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a7df4609-d687-4773-b5b3-a8184d925b6b req-70c5a9ce-d503-4485-993e-42a595cb6c60 service nova] Lock "06481254-722a-4053-a6af-11f5dcc68636-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG nova.compute.manager [req-a7df4609-d687-4773-b5b3-a8184d925b6b req-70c5a9ce-d503-4485-993e-42a595cb6c60 service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] No waiting events found dispatching network-vif-unplugged-5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG nova.compute.manager [req-a7df4609-d687-4773-b5b3-a8184d925b6b req-70c5a9ce-d503-4485-993e-42a595cb6c60 service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Received event network-vif-unplugged-5861aafc-4312-4d82-965c-2d1d993d8ea0 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:08 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Instance destroyed successfully.
Apr 12 03:43:08 user nova-compute[72081]: DEBUG nova.objects.instance [None req-cd85aefe-22ee-42f6-8afe-be96d01ae05f tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lazy-loading 'resources' on Instance uuid 06481254-722a-4053-a6af-11f5dcc68636 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-cd85aefe-22ee-42f6-8afe-be96d01ae05f tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-12T03:37:30Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-405065337',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-405065337',id=20,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEV0Ykds0yE0AtGhC79nOfa8CiixAX+fWYFS4r0KbDrdX/eto3bHX8KjkQpgbhQK4OVD9q33q1+I1pg47ZtrC8YTjrFvz5r71kp0Qd9KtUm3DHiDVGrOt+n/RWN+EIIG+w==',key_name='tempest-keypair-2125155953',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:41:28Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='738ebb6b798c4974aa46f9bbe3828279',ramdisk_id='',reservation_id='r-jbxvhjdl',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-2139975165',owner_user_name='tempest-AttachVolumeShelveTestJSON-2139975165-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:41:28Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='de1fe5db01624224ab7b615c908d990b',uuid=06481254-722a-4053-a6af-11f5dcc68636,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "address": "fa:16:3e:c7:7c:eb", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.227", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5861aafc-43", "ovs_interfaceid": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-cd85aefe-22ee-42f6-8afe-be96d01ae05f tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Converting VIF {"id": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "address": "fa:16:3e:c7:7c:eb", "network": {"id": "638ee582-4cac-44e7-b703-386f5116e8a3", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-327696981-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.227", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "738ebb6b798c4974aa46f9bbe3828279", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5861aafc-43", "ovs_interfaceid": "5861aafc-4312-4d82-965c-2d1d993d8ea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-cd85aefe-22ee-42f6-8afe-be96d01ae05f tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:c7:7c:eb,bridge_name='br-int',has_traffic_filtering=True,id=5861aafc-4312-4d82-965c-2d1d993d8ea0,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5861aafc-43') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG os_vif [None req-cd85aefe-22ee-42f6-8afe-be96d01ae05f tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:c7:7c:eb,bridge_name='br-int',has_traffic_filtering=True,id=5861aafc-4312-4d82-965c-2d1d993d8ea0,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5861aafc-43') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap5861aafc-43, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:43:08 user nova-compute[72081]: INFO os_vif [None req-cd85aefe-22ee-42f6-8afe-be96d01ae05f tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:c7:7c:eb,bridge_name='br-int',has_traffic_filtering=True,id=5861aafc-4312-4d82-965c-2d1d993d8ea0,network=Network(638ee582-4cac-44e7-b703-386f5116e8a3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5861aafc-43')
Apr 12 03:43:08 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-cd85aefe-22ee-42f6-8afe-be96d01ae05f tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Deleting instance files /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636_del
Apr 12 03:43:08 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-cd85aefe-22ee-42f6-8afe-be96d01ae05f tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Deletion of /opt/stack/data/nova/instances/06481254-722a-4053-a6af-11f5dcc68636_del complete
Apr 12 03:43:08 user nova-compute[72081]: INFO nova.compute.manager [None req-cd85aefe-22ee-42f6-8afe-be96d01ae05f tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Took 0.86 seconds to destroy the instance on the hypervisor.
Apr 12 03:43:08 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-cd85aefe-22ee-42f6-8afe-be96d01ae05f tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:43:08 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 06481254-722a-4053-a6af-11f5dcc68636] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:43:09 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:43:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:09 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:43:09 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:43:09 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Took 0.76 seconds to deallocate network for instance.
Apr 12 03:43:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-cd85aefe-22ee-42f6-8afe-be96d01ae05f tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-cd85aefe-22ee-42f6-8afe-be96d01ae05f tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdg (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:43:10 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-cd85aefe-22ee-42f6-8afe-be96d01ae05f tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:43:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:10 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-cd85aefe-22ee-42f6-8afe-be96d01ae05f tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:43:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-cd85aefe-22ee-42f6-8afe-be96d01ae05f tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.464s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:10 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-cd85aefe-22ee-42f6-8afe-be96d01ae05f tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Deleted allocations for instance 06481254-722a-4053-a6af-11f5dcc68636
Apr 12 03:43:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-cd85aefe-22ee-42f6-8afe-be96d01ae05f tempest-AttachVolumeShelveTestJSON-2139975165 tempest-AttachVolumeShelveTestJSON-2139975165-project-member] Lock "06481254-722a-4053-a6af-11f5dcc68636" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.271s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:10 user nova-compute[72081]: DEBUG nova.compute.manager [req-88ef4302-4ba1-476d-8ec4-877ece6b1c49 req-99e539c5-3569-4b75-8906-6ff253356abb service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Received event network-vif-plugged-5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:43:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-88ef4302-4ba1-476d-8ec4-877ece6b1c49 req-99e539c5-3569-4b75-8906-6ff253356abb service nova] Acquiring lock "06481254-722a-4053-a6af-11f5dcc68636-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-88ef4302-4ba1-476d-8ec4-877ece6b1c49 req-99e539c5-3569-4b75-8906-6ff253356abb service nova] Lock "06481254-722a-4053-a6af-11f5dcc68636-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-88ef4302-4ba1-476d-8ec4-877ece6b1c49 req-99e539c5-3569-4b75-8906-6ff253356abb service nova] Lock "06481254-722a-4053-a6af-11f5dcc68636-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:10 user nova-compute[72081]: DEBUG nova.compute.manager [req-88ef4302-4ba1-476d-8ec4-877ece6b1c49 req-99e539c5-3569-4b75-8906-6ff253356abb service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] No waiting events found dispatching network-vif-plugged-5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:43:10 user nova-compute[72081]: WARNING nova.compute.manager [req-88ef4302-4ba1-476d-8ec4-877ece6b1c49 req-99e539c5-3569-4b75-8906-6ff253356abb service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Received unexpected event network-vif-plugged-5861aafc-4312-4d82-965c-2d1d993d8ea0 for instance with vm_state deleted and task_state None.
Apr 12 03:43:10 user nova-compute[72081]: DEBUG nova.compute.manager [req-88ef4302-4ba1-476d-8ec4-877ece6b1c49 req-99e539c5-3569-4b75-8906-6ff253356abb service nova] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Received event network-vif-deleted-5861aafc-4312-4d82-965c-2d1d993d8ea0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:43:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json" returned: 0 in 0.158s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f88ecf99-d158-4f3c-8a87-6c0322fdc8ab/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f88ecf99-d158-4f3c-8a87-6c0322fdc8ab/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f88ecf99-d158-4f3c-8a87-6c0322fdc8ab/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f88ecf99-d158-4f3c-8a87-6c0322fdc8ab/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdf (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:43:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:43:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdd (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:43:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:43:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:12 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:43:12 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:43:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=7756MB free_disk=25.95650863647461GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:43:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Applying migration context for instance 48977eb2-ce43-4577-a1cb-340e35703dbd as it has an incoming, in-progress migration 35efd868-6d17-47a5-bc6f-dc21edd02c98. Migration status is finished {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:950}}
Apr 12 03:43:12 user nova-compute[72081]: INFO nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Updating resource usage from migration 35efd868-6d17-47a5-bc6f-dc21edd02c98
Apr 12 03:43:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 2d0904c3-74e0-47bb-a832-758a7035b2f5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:43:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance bcd267d2-5b2c-4ed2-b634-0210fae94dfd actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:43:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance f73fa883-75f0-4593-a89c-d009c6d4ebf0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:43:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 3d4407f4-e078-4777-a110-cb3f922be2e8 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:43:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:43:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance f88ecf99-d158-4f3c-8a87-6c0322fdc8ab actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:43:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 614495f2-f108-4759-be60-bc6cc689c5b7 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:43:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 606425bb-8968-41f9-b3a5-80b6fd188a67 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:43:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Migration 35efd868-6d17-47a5-bc6f-dc21edd02c98 is active on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 128, 'DISK_GB': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1637}}
Apr 12 03:43:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 48977eb2-ce43-4577-a1cb-340e35703dbd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 192, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:43:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 10 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:43:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=1856MB phys_disk=40GB used_disk=8GB total_vcpus=12 used_vcpus=10 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:43:12 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:43:12 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:43:12 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:43:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.435s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:13 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:43:13 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:43:13 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:43:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:43:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:43:13 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 03:43:14 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Updating instance_info_cache with network_info: [{"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:43:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:43:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 03:43:14 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:43:14 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:43:14 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:43:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:22 user nova-compute[72081]: INFO nova.compute.manager [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Detaching volume 492e66f2-98d6-43d6-bbe6-cbe61f9335f6
Apr 12 03:43:22 user nova-compute[72081]: INFO nova.virt.block_device [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Attempting to driver detach volume 492e66f2-98d6-43d6-bbe6-cbe61f9335f6 from mountpoint /dev/vdb
Apr 12 03:43:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Attempting to detach device vdb from instance f73fa883-75f0-4593-a89c-d009c6d4ebf0 from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 03:43:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:43:22 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:43:22 user nova-compute[72081]:   <source dev="/dev/sdd"/>
Apr 12 03:43:22 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:43:22 user nova-compute[72081]:   <serial>492e66f2-98d6-43d6-bbe6-cbe61f9335f6</serial>
Apr 12 03:43:22 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 03:43:22 user nova-compute[72081]: </disk>
Apr 12 03:43:22 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:43:22 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Successfully detached device vdb from instance f73fa883-75f0-4593-a89c-d009c6d4ebf0 from the persistent domain config.
Apr 12 03:43:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance f73fa883-75f0-4593-a89c-d009c6d4ebf0 from the live domain config. {{(pid=72081) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 12 03:43:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:43:22 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:43:22 user nova-compute[72081]:   <source dev="/dev/sdd"/>
Apr 12 03:43:22 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:43:22 user nova-compute[72081]:   <serial>492e66f2-98d6-43d6-bbe6-cbe61f9335f6</serial>
Apr 12 03:43:22 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 03:43:22 user nova-compute[72081]: </disk>
Apr 12 03:43:22 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Received event <DeviceRemovedEvent: 1681271003.1672053, f73fa883-75f0-4593-a89c-d009c6d4ebf0 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72081) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance f73fa883-75f0-4593-a89c-d009c6d4ebf0 {{(pid=72081) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 12 03:43:23 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Successfully detached device vdb from instance f73fa883-75f0-4593-a89c-d009c6d4ebf0 from the live domain config.
Apr 12 03:43:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'EMwRIslb', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:43:23 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:43:23 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5b16308d-37e2-4400-abce-2899d0cfed83]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:43:23 user nova-compute[72081]: No targets found.
Apr 12 03:43:23 user nova-compute[72081]: iSNS:
Apr 12 03:43:23 user nova-compute[72081]: No targets found.
Apr 12 03:43:23 user nova-compute[72081]: STATIC:
Apr 12 03:43:23 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:43:23 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:43:23 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:43:23 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e
Apr 12 03:43:23 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:43:23 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:43:23 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3
Apr 12 03:43:23 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:43:23 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:43:23 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742
Apr 12 03:43:23 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:43:23 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:43:23 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6
Apr 12 03:43:23 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:43:23 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:43:23 user nova-compute[72081]: FIRMWARE:
Apr 12 03:43:23 user nova-compute[72081]: No targets found.
Apr 12 03:43:23 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:43:23 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:43:23 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1c470d70-2dd0-4525-90ad-7c6863201d4c]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.036s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3a5d9933-4da8-4a73-a410-57546bc03501]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:43:23 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:43:23 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:43:23 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:43:23 user nova-compute[72081]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:43:23 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:43:23 user nova-compute[72081]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 (non-flash)
Apr 12 03:43:23 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:43:23 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:43:23 user nova-compute[72081]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:43:23 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fba3827a0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6'): ({'sdd'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Removing single pathed devices sdd {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.031s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[245724da-9725-4d62-8061-f6d221e18f1a]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.031s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2fd31cd-c72e-4a32-a89b-23505977d8a6]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Flushing IO for device /dev/sdd {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[87c0901d-4883-498c-81ab-68250cea7065]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.056s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6fdbfc4b-9dfc-4d17-a1e4-19593e632426]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] SCSI volumes sdd have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ea3670d8-692a-4f23-97e9-46aac6af5845]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 -p 172.16.0.220:3260 --logout" returned: 0 in 0.061s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[48f5a27a-82a5-41df-9dcb-d220a056ac86]: (4, ('Logging out of session [sid: 27, target: iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6, portal: 172.16.0.220,3260]\nLogout of [sid: 27, target: iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 27, target: iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6, portal: 172.16.0.220,3260]
Apr 12 03:43:23 user nova-compute[72081]: Logout of [sid: 27, target: iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6, portal: 172.16.0.220,3260] successful.
Apr 12 03:43:23 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492e66f2-98d6-43d6-bbe6-cbe61f9335f6 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.036s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bdd01544-0d21-48ba-8eb8-94f85406987a]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.398s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] <== disconnect_volume: return (399ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681270988.8495011, 06481254-722a-4053-a6af-11f5dcc68636 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:43:23 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 06481254-722a-4053-a6af-11f5dcc68636] VM Stopped (Lifecycle Event)
Apr 12 03:43:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-74d013db-23b4-4eaa-a2a9-670778d34c9d None None] [instance: 06481254-722a-4053-a6af-11f5dcc68636] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:43:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:24 user nova-compute[72081]: DEBUG nova.objects.instance [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lazy-loading 'flavor' on Instance uuid f73fa883-75f0-4593-a89c-d009c6d4ebf0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:43:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-920e77f7-9355-483d-8258-8aa3e6563163 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 2.154s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-09412312-e31b-4c99-b552-589337afa498 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Acquiring lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-09412312-e31b-4c99-b552-589337afa498 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-09412312-e31b-4c99-b552-589337afa498 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Acquiring lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-09412312-e31b-4c99-b552-589337afa498 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-09412312-e31b-4c99-b552-589337afa498 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:24 user nova-compute[72081]: INFO nova.compute.manager [None req-09412312-e31b-4c99-b552-589337afa498 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Terminating instance
Apr 12 03:43:24 user nova-compute[72081]: DEBUG nova.compute.manager [None req-09412312-e31b-4c99-b552-589337afa498 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:43:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:25 user nova-compute[72081]: DEBUG nova.compute.manager [req-549b06ca-3cf2-44ca-80ad-a4e1583ad56d req-6d73bd12-0865-42b9-93ae-7bf7f9e2f3d4 service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Received event network-vif-unplugged-1c1d30d9-7f27-4302-bccb-f30b9bf2a36f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:43:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-549b06ca-3cf2-44ca-80ad-a4e1583ad56d req-6d73bd12-0865-42b9-93ae-7bf7f9e2f3d4 service nova] Acquiring lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-549b06ca-3cf2-44ca-80ad-a4e1583ad56d req-6d73bd12-0865-42b9-93ae-7bf7f9e2f3d4 service nova] Lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-549b06ca-3cf2-44ca-80ad-a4e1583ad56d req-6d73bd12-0865-42b9-93ae-7bf7f9e2f3d4 service nova] Lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:25 user nova-compute[72081]: DEBUG nova.compute.manager [req-549b06ca-3cf2-44ca-80ad-a4e1583ad56d req-6d73bd12-0865-42b9-93ae-7bf7f9e2f3d4 service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] No waiting events found dispatching network-vif-unplugged-1c1d30d9-7f27-4302-bccb-f30b9bf2a36f {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:43:25 user nova-compute[72081]: DEBUG nova.compute.manager [req-549b06ca-3cf2-44ca-80ad-a4e1583ad56d req-6d73bd12-0865-42b9-93ae-7bf7f9e2f3d4 service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Received event network-vif-unplugged-1c1d30d9-7f27-4302-bccb-f30b9bf2a36f for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:43:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:25 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Instance destroyed successfully.
Apr 12 03:43:25 user nova-compute[72081]: DEBUG nova.objects.instance [None req-09412312-e31b-4c99-b552-589337afa498 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lazy-loading 'resources' on Instance uuid f88ecf99-d158-4f3c-8a87-6c0322fdc8ab {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:43:25 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-09412312-e31b-4c99-b552-589337afa498 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:41:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-1538366054',display_name='tempest-VolumesActionsTest-instance-1538366054',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-1538366054',id=29,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:41:39Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='f4f984af8c7048c7ba68c1530be9cc8d',ramdisk_id='',reservation_id='r-gkg6fino',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesActionsTest-629816748',owner_user_name='tempest-VolumesActionsTest-629816748-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:41:40Z,user_data=None,user_id='45fe6fed1a5a43e5bd0925b92486ec68',uuid=f88ecf99-d158-4f3c-8a87-6c0322fdc8ab,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "1c1d30d9-7f27-4302-bccb-f30b9bf2a36f", "address": "fa:16:3e:74:3e:6a", "network": {"id": "3ec59075-3409-4372-a0ca-6283a5d56ca0", "bridge": "br-int", "label": "tempest-VolumesActionsTest-959291386-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f4f984af8c7048c7ba68c1530be9cc8d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1c1d30d9-7f", "ovs_interfaceid": "1c1d30d9-7f27-4302-bccb-f30b9bf2a36f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:43:25 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-09412312-e31b-4c99-b552-589337afa498 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Converting VIF {"id": "1c1d30d9-7f27-4302-bccb-f30b9bf2a36f", "address": "fa:16:3e:74:3e:6a", "network": {"id": "3ec59075-3409-4372-a0ca-6283a5d56ca0", "bridge": "br-int", "label": "tempest-VolumesActionsTest-959291386-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f4f984af8c7048c7ba68c1530be9cc8d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1c1d30d9-7f", "ovs_interfaceid": "1c1d30d9-7f27-4302-bccb-f30b9bf2a36f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:43:25 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-09412312-e31b-4c99-b552-589337afa498 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:74:3e:6a,bridge_name='br-int',has_traffic_filtering=True,id=1c1d30d9-7f27-4302-bccb-f30b9bf2a36f,network=Network(3ec59075-3409-4372-a0ca-6283a5d56ca0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1c1d30d9-7f') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:43:25 user nova-compute[72081]: DEBUG os_vif [None req-09412312-e31b-4c99-b552-589337afa498 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:74:3e:6a,bridge_name='br-int',has_traffic_filtering=True,id=1c1d30d9-7f27-4302-bccb-f30b9bf2a36f,network=Network(3ec59075-3409-4372-a0ca-6283a5d56ca0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1c1d30d9-7f') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:43:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap1c1d30d9-7f, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:43:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:25 user nova-compute[72081]: INFO os_vif [None req-09412312-e31b-4c99-b552-589337afa498 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:74:3e:6a,bridge_name='br-int',has_traffic_filtering=True,id=1c1d30d9-7f27-4302-bccb-f30b9bf2a36f,network=Network(3ec59075-3409-4372-a0ca-6283a5d56ca0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1c1d30d9-7f')
Apr 12 03:43:25 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-09412312-e31b-4c99-b552-589337afa498 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Deleting instance files /opt/stack/data/nova/instances/f88ecf99-d158-4f3c-8a87-6c0322fdc8ab_del
Apr 12 03:43:25 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-09412312-e31b-4c99-b552-589337afa498 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Deletion of /opt/stack/data/nova/instances/f88ecf99-d158-4f3c-8a87-6c0322fdc8ab_del complete
Apr 12 03:43:25 user nova-compute[72081]: INFO nova.compute.manager [None req-09412312-e31b-4c99-b552-589337afa498 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Took 0.71 seconds to destroy the instance on the hypervisor.
Apr 12 03:43:25 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-09412312-e31b-4c99-b552-589337afa498 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:43:25 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:43:25 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:43:27 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Took 1.61 seconds to deallocate network for instance.
Apr 12 03:43:27 user nova-compute[72081]: DEBUG nova.compute.manager [req-729aa739-471e-483a-8a14-708934d13118 req-04e084e7-4e9b-4c8d-af3e-4539f7a5ba73 service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Received event network-vif-plugged-1c1d30d9-7f27-4302-bccb-f30b9bf2a36f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-729aa739-471e-483a-8a14-708934d13118 req-04e084e7-4e9b-4c8d-af3e-4539f7a5ba73 service nova] Acquiring lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-729aa739-471e-483a-8a14-708934d13118 req-04e084e7-4e9b-4c8d-af3e-4539f7a5ba73 service nova] Lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-729aa739-471e-483a-8a14-708934d13118 req-04e084e7-4e9b-4c8d-af3e-4539f7a5ba73 service nova] Lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG nova.compute.manager [req-729aa739-471e-483a-8a14-708934d13118 req-04e084e7-4e9b-4c8d-af3e-4539f7a5ba73 service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] No waiting events found dispatching network-vif-plugged-1c1d30d9-7f27-4302-bccb-f30b9bf2a36f {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:43:27 user nova-compute[72081]: WARNING nova.compute.manager [req-729aa739-471e-483a-8a14-708934d13118 req-04e084e7-4e9b-4c8d-af3e-4539f7a5ba73 service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Received unexpected event network-vif-plugged-1c1d30d9-7f27-4302-bccb-f30b9bf2a36f for instance with vm_state active and task_state deleting.
Apr 12 03:43:27 user nova-compute[72081]: DEBUG nova.compute.manager [req-729aa739-471e-483a-8a14-708934d13118 req-04e084e7-4e9b-4c8d-af3e-4539f7a5ba73 service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Received event network-vif-plugged-1c1d30d9-7f27-4302-bccb-f30b9bf2a36f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-729aa739-471e-483a-8a14-708934d13118 req-04e084e7-4e9b-4c8d-af3e-4539f7a5ba73 service nova] Acquiring lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-729aa739-471e-483a-8a14-708934d13118 req-04e084e7-4e9b-4c8d-af3e-4539f7a5ba73 service nova] Lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-729aa739-471e-483a-8a14-708934d13118 req-04e084e7-4e9b-4c8d-af3e-4539f7a5ba73 service nova] Lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG nova.compute.manager [req-729aa739-471e-483a-8a14-708934d13118 req-04e084e7-4e9b-4c8d-af3e-4539f7a5ba73 service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] No waiting events found dispatching network-vif-plugged-1c1d30d9-7f27-4302-bccb-f30b9bf2a36f {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:43:27 user nova-compute[72081]: WARNING nova.compute.manager [req-729aa739-471e-483a-8a14-708934d13118 req-04e084e7-4e9b-4c8d-af3e-4539f7a5ba73 service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Received unexpected event network-vif-plugged-1c1d30d9-7f27-4302-bccb-f30b9bf2a36f for instance with vm_state active and task_state deleting.
Apr 12 03:43:27 user nova-compute[72081]: DEBUG nova.compute.manager [req-729aa739-471e-483a-8a14-708934d13118 req-04e084e7-4e9b-4c8d-af3e-4539f7a5ba73 service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Received event network-vif-plugged-1c1d30d9-7f27-4302-bccb-f30b9bf2a36f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-729aa739-471e-483a-8a14-708934d13118 req-04e084e7-4e9b-4c8d-af3e-4539f7a5ba73 service nova] Acquiring lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-729aa739-471e-483a-8a14-708934d13118 req-04e084e7-4e9b-4c8d-af3e-4539f7a5ba73 service nova] Lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-729aa739-471e-483a-8a14-708934d13118 req-04e084e7-4e9b-4c8d-af3e-4539f7a5ba73 service nova] Lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG nova.compute.manager [req-729aa739-471e-483a-8a14-708934d13118 req-04e084e7-4e9b-4c8d-af3e-4539f7a5ba73 service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] No waiting events found dispatching network-vif-plugged-1c1d30d9-7f27-4302-bccb-f30b9bf2a36f {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:43:27 user nova-compute[72081]: WARNING nova.compute.manager [req-729aa739-471e-483a-8a14-708934d13118 req-04e084e7-4e9b-4c8d-af3e-4539f7a5ba73 service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Received unexpected event network-vif-plugged-1c1d30d9-7f27-4302-bccb-f30b9bf2a36f for instance with vm_state active and task_state deleting.
Apr 12 03:43:27 user nova-compute[72081]: DEBUG nova.compute.manager [req-729aa739-471e-483a-8a14-708934d13118 req-04e084e7-4e9b-4c8d-af3e-4539f7a5ba73 service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Received event network-vif-plugged-1c1d30d9-7f27-4302-bccb-f30b9bf2a36f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-729aa739-471e-483a-8a14-708934d13118 req-04e084e7-4e9b-4c8d-af3e-4539f7a5ba73 service nova] Acquiring lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-729aa739-471e-483a-8a14-708934d13118 req-04e084e7-4e9b-4c8d-af3e-4539f7a5ba73 service nova] Lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-729aa739-471e-483a-8a14-708934d13118 req-04e084e7-4e9b-4c8d-af3e-4539f7a5ba73 service nova] Lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG nova.compute.manager [req-729aa739-471e-483a-8a14-708934d13118 req-04e084e7-4e9b-4c8d-af3e-4539f7a5ba73 service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] No waiting events found dispatching network-vif-plugged-1c1d30d9-7f27-4302-bccb-f30b9bf2a36f {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:43:27 user nova-compute[72081]: WARNING nova.compute.manager [req-729aa739-471e-483a-8a14-708934d13118 req-04e084e7-4e9b-4c8d-af3e-4539f7a5ba73 service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Received unexpected event network-vif-plugged-1c1d30d9-7f27-4302-bccb-f30b9bf2a36f for instance with vm_state active and task_state deleting.
Apr 12 03:43:27 user nova-compute[72081]: DEBUG nova.compute.manager [req-729aa739-471e-483a-8a14-708934d13118 req-04e084e7-4e9b-4c8d-af3e-4539f7a5ba73 service nova] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Received event network-vif-deleted-1c1d30d9-7f27-4302-bccb-f30b9bf2a36f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-09412312-e31b-4c99-b552-589337afa498 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-09412312-e31b-4c99-b552-589337afa498 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-09412312-e31b-4c99-b552-589337afa498 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-09412312-e31b-4c99-b552-589337afa498 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-09412312-e31b-4c99-b552-589337afa498 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.345s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:27 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-09412312-e31b-4c99-b552-589337afa498 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Deleted allocations for instance f88ecf99-d158-4f3c-8a87-6c0322fdc8ab
Apr 12 03:43:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-09412312-e31b-4c99-b552-589337afa498 tempest-VolumesActionsTest-629816748 tempest-VolumesActionsTest-629816748-project-member] Lock "f88ecf99-d158-4f3c-8a87-6c0322fdc8ab" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.862s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "93e45116-bb5c-40ca-9fc5-1c5c40ff687e" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "93e45116-bb5c-40ca-9fc5-1c5c40ff687e" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:27 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:43:27 user nova-compute[72081]: INFO nova.compute.claims [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Claim successful on node user
Apr 12 03:43:28 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.392s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:43:28 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:43:28 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:43:28 user nova-compute[72081]: INFO nova.virt.block_device [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Booting with volume ac69f59b-6bc4-4f63-bbf6-2952889fcf27 at /dev/vda
Apr 12 03:43:28 user nova-compute[72081]: DEBUG nova.policy [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'b00b348933224558aad1cb12921f100f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '003688353e2f4b36831c574ebaf65173', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG os_brick.utils [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[55f46087-4142-49fc-8474-e02a16e98310]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e5dd4698-a7ab-4f16-93d3-2e7dccdc2c0f]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[049aabcd-3cdf-4a07-bffe-acb4885a0c00]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e4a060e8-e2f0-462e-ba6d-d12e4c7298cc]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:43:28 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:43:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0557a4cb-75fa-464a-bbcc-a21bbc7c72ad]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG os_brick.utils [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] <== get_connector_properties: return (144ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:43:28 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Updating existing volume attachment record: 9c661fe8-f2d6-4938-8bf7-8bd8be946a16 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:43:29 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Successfully created port: e210f496-084e-443f-9c7f-684661441f6c {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:43:29 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Successfully updated port: e210f496-084e-443f-9c7f-684661441f6c {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:43:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "refresh_cache-93e45116-bb5c-40ca-9fc5-1c5c40ff687e" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:43:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquired lock "refresh_cache-93e45116-bb5c-40ca-9fc5-1c5c40ff687e" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:43:30 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:43:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-edbe44e3-0eae-4b1d-892e-ea38f800ee33 req-4d0f897d-5fee-453c-8191-a123a47af26d service nova] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Received event network-changed-e210f496-084e-443f-9c7f-684661441f6c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:43:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-edbe44e3-0eae-4b1d-892e-ea38f800ee33 req-4d0f897d-5fee-453c-8191-a123a47af26d service nova] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Refreshing instance network info cache due to event network-changed-e210f496-084e-443f-9c7f-684661441f6c. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:43:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-edbe44e3-0eae-4b1d-892e-ea38f800ee33 req-4d0f897d-5fee-453c-8191-a123a47af26d service nova] Acquiring lock "refresh_cache-93e45116-bb5c-40ca-9fc5-1c5c40ff687e" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:43:30 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:43:30 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Updating instance_info_cache with network_info: [{"id": "e210f496-084e-443f-9c7f-684661441f6c", "address": "fa:16:3e:9e:2a:c9", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape210f496-08", "ovs_interfaceid": "e210f496-084e-443f-9c7f-684661441f6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:43:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Releasing lock "refresh_cache-93e45116-bb5c-40ca-9fc5-1c5c40ff687e" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:43:30 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Instance network_info: |[{"id": "e210f496-084e-443f-9c7f-684661441f6c", "address": "fa:16:3e:9e:2a:c9", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape210f496-08", "ovs_interfaceid": "e210f496-084e-443f-9c7f-684661441f6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:43:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-edbe44e3-0eae-4b1d-892e-ea38f800ee33 req-4d0f897d-5fee-453c-8191-a123a47af26d service nova] Acquired lock "refresh_cache-93e45116-bb5c-40ca-9fc5-1c5c40ff687e" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:43:30 user nova-compute[72081]: DEBUG nova.network.neutron [req-edbe44e3-0eae-4b1d-892e-ea38f800ee33 req-4d0f897d-5fee-453c-8191-a123a47af26d service nova] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Refreshing network info cache for port e210f496-084e-443f-9c7f-684661441f6c {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:43:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:30 user nova-compute[72081]: DEBUG nova.network.neutron [req-edbe44e3-0eae-4b1d-892e-ea38f800ee33 req-4d0f897d-5fee-453c-8191-a123a47af26d service nova] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Updated VIF entry in instance network info cache for port e210f496-084e-443f-9c7f-684661441f6c. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:43:30 user nova-compute[72081]: DEBUG nova.network.neutron [req-edbe44e3-0eae-4b1d-892e-ea38f800ee33 req-4d0f897d-5fee-453c-8191-a123a47af26d service nova] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Updating instance_info_cache with network_info: [{"id": "e210f496-084e-443f-9c7f-684661441f6c", "address": "fa:16:3e:9e:2a:c9", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape210f496-08", "ovs_interfaceid": "e210f496-084e-443f-9c7f-684661441f6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:43:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-edbe44e3-0eae-4b1d-892e-ea38f800ee33 req-4d0f897d-5fee-453c-8191-a123a47af26d service nova] Releasing lock "refresh_cache-93e45116-bb5c-40ca-9fc5-1c5c40ff687e" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:43:32 user nova-compute[72081]: INFO nova.compute.manager [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Rescuing
Apr 12 03:43:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "refresh_cache-614495f2-f108-4759-be60-bc6cc689c5b7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:43:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquired lock "refresh_cache-614495f2-f108-4759-be60-bc6cc689c5b7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:43:32 user nova-compute[72081]: DEBUG nova.network.neutron [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG nova.network.neutron [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Updating instance_info_cache with network_info: [{"id": "b9acaee6-a684-4827-bc91-1c6219782efe", "address": "fa:16:3e:d4:dc:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb9acaee6-a6", "ovs_interfaceid": "b9acaee6-a684-4827-bc91-1c6219782efe", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:43:33 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Creating image(s)
Apr 12 03:43:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Did not create local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Ensure instance console log exists: /opt/stack/data/nova/instances/93e45116-bb5c-40ca-9fc5-1c5c40ff687e/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Start _get_guest_xml network_info=[{"id": "e210f496-084e-443f-9c7f-684661441f6c", "address": "fa:16:3e:9e:2a:c9", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape210f496-08", "ovs_interfaceid": "e210f496-084e-443f-9c7f-684661441f6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wmlvgqFk', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '93e45116-bb5c-40ca-9fc5-1c5c40ff687e', 'attached_at': '', 'detached_at': '', 'volume_id': 'ac69f59b-6bc4-4f63-bbf6-2952889fcf27', 'serial': 'ac69f59b-6bc4-4f63-bbf6-2952889fcf27'}, 'disk_bus': 'virtio', 'delete_on_termination': False, 'boot_index': 0, 'guest_format': None, 'device_type': 'disk', 'attachment_id': '9c661fe8-f2d6-4938-8bf7-8bd8be946a16', 'volume_type': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Releasing lock "refresh_cache-614495f2-f108-4759-be60-bc6cc689c5b7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:43:33 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:43:33 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:43:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wmlvgqFk', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:43:33 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:43:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:43:33 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:43:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:43:33 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2dbb50dd-a3ab-4570-a68a-5b1eb5f2ff36]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:43:33 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d5606b32-c317-4ff6-84a0-2033a60f5478]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27] added
Apr 12 03:43:33 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2d0dbd47-bcfe-44c9-b38f-504a54607c4e]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ad10a725-da00-4bcf-b78e-770200d27d07]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v wmlvgqFk {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v wmlvgqFk" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[31303da8-91c1-40b7-98e0-aa1a7e3bbc41]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'wmlvgqFk'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[45e47e40-d7eb-4e1b-b6fb-624b15e20bda]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6cf3cf87-0793-46d8-b4c3-8454ecee7902]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:43:33 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:43:33 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:43:33 user nova-compute[72081]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:43:33 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:43:33 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:43:33 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:43:33 user nova-compute[72081]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:43:33 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --login" returned: 0 in 0.072s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ba213091-8905-4da9-92a1-7986778c2b3f]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27, portal: 172.16.0.220,3260]
Apr 12 03:43:33 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27, portal: 172.16.0.220,3260] successful.
Apr 12 03:43:33 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-a59ee089-6d09-43ad-b3db-d0c0bc0eced0 req-4e54708d-7dce-4d9b-b8c6-5dee206cd7a8 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received event network-vif-unplugged-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a59ee089-6d09-43ad-b3db-d0c0bc0eced0 req-4e54708d-7dce-4d9b-b8c6-5dee206cd7a8 service nova] Acquiring lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a59ee089-6d09-43ad-b3db-d0c0bc0eced0 req-4e54708d-7dce-4d9b-b8c6-5dee206cd7a8 service nova] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a59ee089-6d09-43ad-b3db-d0c0bc0eced0 req-4e54708d-7dce-4d9b-b8c6-5dee206cd7a8 service nova] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG nova.compute.manager [req-a59ee089-6d09-43ad-b3db-d0c0bc0eced0 req-4e54708d-7dce-4d9b-b8c6-5dee206cd7a8 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] No waiting events found dispatching network-vif-unplugged-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:43:33 user nova-compute[72081]: WARNING nova.compute.manager [req-a59ee089-6d09-43ad-b3db-d0c0bc0eced0 req-4e54708d-7dce-4d9b-b8c6-5dee206cd7a8 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received unexpected event network-vif-unplugged-b9acaee6-a684-4827-bc91-1c6219782efe for instance with vm_state active and task_state rescuing.
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.099s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e1a1725b-8f28-4c55-b1ad-092d8accdfbb]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.037s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[30511dd5-41c1-4baa-8f93-e689bb7f7e6f]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:43:33 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:43:33 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:43:33 user nova-compute[72081]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:43:33 user nova-compute[72081]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:43:33 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:43:33 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:43:33 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:43:33 user nova-compute[72081]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:43:33 user nova-compute[72081]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:43:33 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.407s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] HCTL ('35', '-', '-', 0) found on session 32 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8881cc89-d465-44c0-b45b-044967f4f748]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Searching for a device in session 32 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:43:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wmlvgqFk', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:43:33 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Instance destroyed successfully.
Apr 12 03:43:33 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Attempting a stable device rescue
Apr 12 03:43:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} {{(pid=72081) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4289}}
Apr 12 03:43:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Instance directory exists: not creating {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4694}}
Apr 12 03:43:34 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Creating image(s)
Apr 12 03:43:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "/opt/stack/data/nova/instances/614495f2-f108-4759-be60-bc6cc689c5b7/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "/opt/stack/data/nova/instances/614495f2-f108-4759-be60-bc6cc689c5b7/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "/opt/stack/data/nova/instances/614495f2-f108-4759-be60-bc6cc689c5b7/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:34 user nova-compute[72081]: DEBUG nova.objects.instance [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lazy-loading 'trusted_certs' on Instance uuid 614495f2-f108-4759-be60-bc6cc689c5b7 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:43:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "1b8169da5e88f5971780e0a8c6222f174cec0901" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "1b8169da5e88f5971780e0a8c6222f174cec0901" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/1b8169da5e88f5971780e0a8c6222f174cec0901.part --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.981s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] <== connect_volume: return (1984ms) {'type': 'block', 'scsi_wwn': '26135356462333539', 'path': '/dev/sdd'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26135356462333539', 'path': '/dev/sdd'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:43:27Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-565513575',display_name='tempest-TestVolumeBootPattern-server-565513575',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-565513575',id=32,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBASnyncVOhHKPzClYx3zwpDcJrieJkBwHOEHWlg2NFTJJdBa1/TdotUOLxC4yYnCsrHxIQy7r8qgZsehiquLQNSYVTnDsRU6DmQUCbMUYEphptEyPo7yf2/NkfI88nssBw==',key_name='tempest-TestVolumeBootPattern-54080024',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='003688353e2f4b36831c574ebaf65173',ramdisk_id='',reservation_id='r-48mm0c6c',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-235043461',owner_user_name='tempest-TestVolumeBootPattern-235043461-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:43:28Z,user_data=None,user_id='b00b348933224558aad1cb12921f100f',uuid=93e45116-bb5c-40ca-9fc5-1c5c40ff687e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e210f496-084e-443f-9c7f-684661441f6c", "address": "fa:16:3e:9e:2a:c9", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape210f496-08", "ovs_interfaceid": "e210f496-084e-443f-9c7f-684661441f6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converting VIF {"id": "e210f496-084e-443f-9c7f-684661441f6c", "address": "fa:16:3e:9e:2a:c9", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape210f496-08", "ovs_interfaceid": "e210f496-084e-443f-9c7f-684661441f6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:9e:2a:c9,bridge_name='br-int',has_traffic_filtering=True,id=e210f496-084e-443f-9c7f-684661441f6c,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape210f496-08') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lazy-loading 'pci_devices' on Instance uuid 93e45116-bb5c-40ca-9fc5-1c5c40ff687e {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:43:35 user nova-compute[72081]:   <uuid>93e45116-bb5c-40ca-9fc5-1c5c40ff687e</uuid>
Apr 12 03:43:35 user nova-compute[72081]:   <name>instance-00000020</name>
Apr 12 03:43:35 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:43:35 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:43:35 user nova-compute[72081]:   <metadata>
Apr 12 03:43:35 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:43:35 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:43:35 user nova-compute[72081]:       <nova:name>tempest-TestVolumeBootPattern-server-565513575</nova:name>
Apr 12 03:43:35 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:43:33</nova:creationTime>
Apr 12 03:43:35 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:43:35 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:43:35 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:43:35 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:43:35 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:43:35 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:43:35 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:43:35 user nova-compute[72081]:       <nova:owner>
Apr 12 03:43:35 user nova-compute[72081]:         <nova:user uuid="b00b348933224558aad1cb12921f100f">tempest-TestVolumeBootPattern-235043461-project-member</nova:user>
Apr 12 03:43:35 user nova-compute[72081]:         <nova:project uuid="003688353e2f4b36831c574ebaf65173">tempest-TestVolumeBootPattern-235043461</nova:project>
Apr 12 03:43:35 user nova-compute[72081]:       </nova:owner>
Apr 12 03:43:35 user nova-compute[72081]:       <nova:ports>
Apr 12 03:43:35 user nova-compute[72081]:         <nova:port uuid="e210f496-084e-443f-9c7f-684661441f6c">
Apr 12 03:43:35 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Apr 12 03:43:35 user nova-compute[72081]:         </nova:port>
Apr 12 03:43:35 user nova-compute[72081]:       </nova:ports>
Apr 12 03:43:35 user nova-compute[72081]:     </nova:instance>
Apr 12 03:43:35 user nova-compute[72081]:   </metadata>
Apr 12 03:43:35 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:43:35 user nova-compute[72081]:     <system>
Apr 12 03:43:35 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:43:35 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:43:35 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:43:35 user nova-compute[72081]:       <entry name="serial">93e45116-bb5c-40ca-9fc5-1c5c40ff687e</entry>
Apr 12 03:43:35 user nova-compute[72081]:       <entry name="uuid">93e45116-bb5c-40ca-9fc5-1c5c40ff687e</entry>
Apr 12 03:43:35 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:43:35 user nova-compute[72081]:     </system>
Apr 12 03:43:35 user nova-compute[72081]:   </sysinfo>
Apr 12 03:43:35 user nova-compute[72081]:   <os>
Apr 12 03:43:35 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:43:35 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:43:35 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:43:35 user nova-compute[72081]:   </os>
Apr 12 03:43:35 user nova-compute[72081]:   <features>
Apr 12 03:43:35 user nova-compute[72081]:     <acpi/>
Apr 12 03:43:35 user nova-compute[72081]:     <apic/>
Apr 12 03:43:35 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:43:35 user nova-compute[72081]:   </features>
Apr 12 03:43:35 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:43:35 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:43:35 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:43:35 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:43:35 user nova-compute[72081]:   </clock>
Apr 12 03:43:35 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:43:35 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:43:35 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:43:35 user nova-compute[72081]:   </cpu>
Apr 12 03:43:35 user nova-compute[72081]:   <devices>
Apr 12 03:43:35 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:43:35 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:43:35 user nova-compute[72081]:       <source dev="/dev/sdd"/>
Apr 12 03:43:35 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:43:35 user nova-compute[72081]:       <serial>ac69f59b-6bc4-4f63-bbf6-2952889fcf27</serial>
Apr 12 03:43:35 user nova-compute[72081]:     </disk>
Apr 12 03:43:35 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:43:35 user nova-compute[72081]:       <mac address="fa:16:3e:9e:2a:c9"/>
Apr 12 03:43:35 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:43:35 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:43:35 user nova-compute[72081]:       <target dev="tape210f496-08"/>
Apr 12 03:43:35 user nova-compute[72081]:     </interface>
Apr 12 03:43:35 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:43:35 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/93e45116-bb5c-40ca-9fc5-1c5c40ff687e/console.log" append="off"/>
Apr 12 03:43:35 user nova-compute[72081]:     </serial>
Apr 12 03:43:35 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:43:35 user nova-compute[72081]:     <video>
Apr 12 03:43:35 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:43:35 user nova-compute[72081]:     </video>
Apr 12 03:43:35 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:43:35 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:43:35 user nova-compute[72081]:     </rng>
Apr 12 03:43:35 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:43:35 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:43:35 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:43:35 user nova-compute[72081]:     </memballoon>
Apr 12 03:43:35 user nova-compute[72081]:   </devices>
Apr 12 03:43:35 user nova-compute[72081]: </domain>
Apr 12 03:43:35 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:43:27Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-565513575',display_name='tempest-TestVolumeBootPattern-server-565513575',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-565513575',id=32,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBASnyncVOhHKPzClYx3zwpDcJrieJkBwHOEHWlg2NFTJJdBa1/TdotUOLxC4yYnCsrHxIQy7r8qgZsehiquLQNSYVTnDsRU6DmQUCbMUYEphptEyPo7yf2/NkfI88nssBw==',key_name='tempest-TestVolumeBootPattern-54080024',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='003688353e2f4b36831c574ebaf65173',ramdisk_id='',reservation_id='r-48mm0c6c',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-235043461',owner_user_name='tempest-TestVolumeBootPattern-235043461-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:43:28Z,user_data=None,user_id='b00b348933224558aad1cb12921f100f',uuid=93e45116-bb5c-40ca-9fc5-1c5c40ff687e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e210f496-084e-443f-9c7f-684661441f6c", "address": "fa:16:3e:9e:2a:c9", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape210f496-08", "ovs_interfaceid": "e210f496-084e-443f-9c7f-684661441f6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converting VIF {"id": "e210f496-084e-443f-9c7f-684661441f6c", "address": "fa:16:3e:9e:2a:c9", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape210f496-08", "ovs_interfaceid": "e210f496-084e-443f-9c7f-684661441f6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:9e:2a:c9,bridge_name='br-int',has_traffic_filtering=True,id=e210f496-084e-443f-9c7f-684661441f6c,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape210f496-08') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG os_vif [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:9e:2a:c9,bridge_name='br-int',has_traffic_filtering=True,id=e210f496-084e-443f-9c7f-684661441f6c,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape210f496-08') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tape210f496-08, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tape210f496-08, col_values=(('external_ids', {'iface-id': 'e210f496-084e-443f-9c7f-684661441f6c', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:9e:2a:c9', 'vm-uuid': '93e45116-bb5c-40ca-9fc5-1c5c40ff687e'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:35 user nova-compute[72081]: INFO os_vif [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:9e:2a:c9,bridge_name='br-int',has_traffic_filtering=True,id=e210f496-084e-443f-9c7f-684661441f6c,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape210f496-08')
Apr 12 03:43:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/1b8169da5e88f5971780e0a8c6222f174cec0901.part --force-share --output=json" returned: 0 in 0.155s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.virt.images [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] 59dbe5d0-117d-453f-a4fa-c324b1b134e0 was qcow2, converting to raw {{(pid=72081) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.privsep.utils [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72081) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/1b8169da5e88f5971780e0a8c6222f174cec0901.part /opt/stack/data/nova/instances/_base/1b8169da5e88f5971780e0a8c6222f174cec0901.converted {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] No VIF found with MAC fa:16:3e:9e:2a:c9, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/1b8169da5e88f5971780e0a8c6222f174cec0901.part /opt/stack/data/nova/instances/_base/1b8169da5e88f5971780e0a8c6222f174cec0901.converted" returned: 0 in 0.286s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/1b8169da5e88f5971780e0a8c6222f174cec0901.converted --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.compute.manager [req-8bb9a64c-d42d-439c-b333-5beeab5570ac req-40b1e2b5-fa6f-4042-90ec-be237ba02107 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received event network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8bb9a64c-d42d-439c-b333-5beeab5570ac req-40b1e2b5-fa6f-4042-90ec-be237ba02107 service nova] Acquiring lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8bb9a64c-d42d-439c-b333-5beeab5570ac req-40b1e2b5-fa6f-4042-90ec-be237ba02107 service nova] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8bb9a64c-d42d-439c-b333-5beeab5570ac req-40b1e2b5-fa6f-4042-90ec-be237ba02107 service nova] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.compute.manager [req-8bb9a64c-d42d-439c-b333-5beeab5570ac req-40b1e2b5-fa6f-4042-90ec-be237ba02107 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] No waiting events found dispatching network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:43:35 user nova-compute[72081]: WARNING nova.compute.manager [req-8bb9a64c-d42d-439c-b333-5beeab5570ac req-40b1e2b5-fa6f-4042-90ec-be237ba02107 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received unexpected event network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe for instance with vm_state active and task_state rescuing.
Apr 12 03:43:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/1b8169da5e88f5971780e0a8c6222f174cec0901.converted --force-share --output=json" returned: 0 in 0.147s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "1b8169da5e88f5971780e0a8c6222f174cec0901" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.670s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "1b8169da5e88f5971780e0a8c6222f174cec0901" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "1b8169da5e88f5971780e0a8c6222f174cec0901" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/1b8169da5e88f5971780e0a8c6222f174cec0901 --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/1b8169da5e88f5971780e0a8c6222f174cec0901 --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/1b8169da5e88f5971780e0a8c6222f174cec0901,backing_fmt=raw /opt/stack/data/nova/instances/614495f2-f108-4759-be60-bc6cc689c5b7/disk.rescue {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/1b8169da5e88f5971780e0a8c6222f174cec0901,backing_fmt=raw /opt/stack/data/nova/instances/614495f2-f108-4759-be60-bc6cc689c5b7/disk.rescue" returned: 0 in 0.050s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "1b8169da5e88f5971780e0a8c6222f174cec0901" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.192s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.objects.instance [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lazy-loading 'migration_context' on Instance uuid 614495f2-f108-4759-be60-bc6cc689c5b7 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Start _get_guest_xml network_info=[{"id": "b9acaee6-a684-4827-bc91-1c6219782efe", "address": "fa:16:3e:d4:dc:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "vif_mac": "fa:16:3e:d4:dc:db"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb9acaee6-a6", "ovs_interfaceid": "b9acaee6-a684-4827-bc91-1c6219782efe", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': '59dbe5d0-117d-453f-a4fa-c324b1b134e0', 'kernel_id': '', 'ramdisk_id': ''} block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'fbzrSZhe', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, 'status': 'reserved', 'instance': '614495f2-f108-4759-be60-bc6cc689c5b7', 'attached_at': '', 'detached_at': '', 'volume_id': '54a9604a-4853-414c-850e-51745d6fcd0e', 'serial': '54a9604a-4853-414c-850e-51745d6fcd0e'}, 'disk_bus': 'virtio', 'delete_on_termination': False, 'boot_index': 0, 'guest_format': None, 'device_type': 'disk', 'attachment_id': 'e53160c2-1524-4d29-8668-bda2166b6071', 'volume_type': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.objects.instance [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lazy-loading 'resources' on Instance uuid 614495f2-f108-4759-be60-bc6cc689c5b7 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.objects.instance [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lazy-loading 'numa_topology' on Instance uuid 614495f2-f108-4759-be60-bc6cc689c5b7 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:43:35 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:43:35 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:43:35 user nova-compute[72081]: DEBUG nova.objects.instance [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lazy-loading 'vcpu_model' on Instance uuid 614495f2-f108-4759-be60-bc6cc689c5b7 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'fbzrSZhe', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:43:36 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:43:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:43:36 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:43:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:43:36 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:43:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[58af61b8-da87-42c0-8fa3-f540a65ea2c4]: (4, ('# BEGIN RECORD 2.1.5\nnode.name = iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = <empty>\niface.ipaddress = <empty>\niface.prefix_len = 0\niface.hwaddress = <empty>\niface.transport_name = tcp\niface.initiatorname = <empty>\niface.state = <empty>\niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = <empty>\niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = <empty>\niface.subnet_mask = <empty>\niface.gateway = <empty>\niface.dhcp_alt_client_id_state = <empty>\niface.dhcp_alt_client_id = <empty>\niface.dhcp_dns = <empty>\niface.dhcp_learn_iqn = <empty>\niface.dhcp_req_vendor_id_state = <empty>\niface.dhcp_vendor_id_state = <empty>\niface.dhcp_vendor_id = <empty>\niface.dhcp_slp_da = <empty>\niface.fragmentation = <empty>\niface.gratuitous_arp = <empty>\niface.incoming_forwarding = <empty>\niface.tos_state = <empty>\niface.tos = 0\niface.ttl = 0\niface.delayed_ack = <empty>\niface.tcp_nagle = <empty>\niface.tcp_wsf_state = <empty>\niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = <empty>\niface.redirect = <empty>\niface.def_task_mgmt_timeout = 0\niface.header_digest = <empty>\niface.data_digest = <empty>\niface.immediate_data = <empty>\niface.initial_r2t = <empty>\niface.data_seq_inorder = <empty>\niface.data_pdu_inorder = <empty>\niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = <empty>\niface.bidi_chap = <empty>\niface.strict_login_compliance = <empty>\niface.discovery_auth = <empty>\niface.discovery_logout = <empty>\nnode.discovery_address = <empty>\nnode.discovery_port = 0\nnode.discovery_type = static\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = -20\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = CHAP\nnode.session.auth.username = fbzrSZhe\nnode.session.auth.password = ********\nnode.session.auth.username_in = <empty>\nnode.session.auth.password_in = <empty>\nnode.session.auth.chap_algs = MD5\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = manual\nnode.session.reopen_max = 0\nnode.conn[0].address = 172.16.0.220\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm (): stdout=# BEGIN RECORD 2.1.5
Apr 12 03:43:36 user nova-compute[72081]: node.name = iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e
Apr 12 03:43:36 user nova-compute[72081]: node.tpgt = -1
Apr 12 03:43:36 user nova-compute[72081]: node.startup = automatic
Apr 12 03:43:36 user nova-compute[72081]: node.leading_login = No
Apr 12 03:43:36 user nova-compute[72081]: iface.iscsi_ifacename = default
Apr 12 03:43:36 user nova-compute[72081]: iface.net_ifacename = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.ipaddress = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.prefix_len = 0
Apr 12 03:43:36 user nova-compute[72081]: iface.hwaddress = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.transport_name = tcp
Apr 12 03:43:36 user nova-compute[72081]: iface.initiatorname = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.state = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.vlan_id = 0
Apr 12 03:43:36 user nova-compute[72081]: iface.vlan_priority = 0
Apr 12 03:43:36 user nova-compute[72081]: iface.vlan_state = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.iface_num = 0
Apr 12 03:43:36 user nova-compute[72081]: iface.mtu = 0
Apr 12 03:43:36 user nova-compute[72081]: iface.port = 0
Apr 12 03:43:36 user nova-compute[72081]: iface.bootproto = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.subnet_mask = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.gateway = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.dhcp_alt_client_id_state = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.dhcp_alt_client_id = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.dhcp_dns = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.dhcp_learn_iqn = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.dhcp_req_vendor_id_state = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.dhcp_vendor_id_state = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.dhcp_vendor_id = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.dhcp_slp_da = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.fragmentation = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.gratuitous_arp = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.incoming_forwarding = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.tos_state = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.tos = 0
Apr 12 03:43:36 user nova-compute[72081]: iface.ttl = 0
Apr 12 03:43:36 user nova-compute[72081]: iface.delayed_ack = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.tcp_nagle = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.tcp_wsf_state = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.tcp_wsf = 0
Apr 12 03:43:36 user nova-compute[72081]: iface.tcp_timer_scale = 0
Apr 12 03:43:36 user nova-compute[72081]: iface.tcp_timestamp = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.redirect = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.def_task_mgmt_timeout = 0
Apr 12 03:43:36 user nova-compute[72081]: iface.header_digest = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.data_digest = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.immediate_data = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.initial_r2t = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.data_seq_inorder = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.data_pdu_inorder = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.erl = 0
Apr 12 03:43:36 user nova-compute[72081]: iface.max_receive_data_len = 0
Apr 12 03:43:36 user nova-compute[72081]: iface.first_burst_len = 0
Apr 12 03:43:36 user nova-compute[72081]: iface.max_outstanding_r2t = 0
Apr 12 03:43:36 user nova-compute[72081]: iface.max_burst_len = 0
Apr 12 03:43:36 user nova-compute[72081]: iface.chap_auth = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.bidi_chap = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.strict_login_compliance = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.discovery_auth = <empty>
Apr 12 03:43:36 user nova-compute[72081]: iface.discovery_logout = <empty>
Apr 12 03:43:36 user nova-compute[72081]: node.discovery_address = <empty>
Apr 12 03:43:36 user nova-compute[72081]: node.discovery_port = 0
Apr 12 03:43:36 user nova-compute[72081]: node.discovery_type = static
Apr 12 03:43:36 user nova-compute[72081]: node.session.initial_cmdsn = 0
Apr 12 03:43:36 user nova-compute[72081]: node.session.initial_login_retry_max = 8
Apr 12 03:43:36 user nova-compute[72081]: node.session.xmit_thread_priority = -20
Apr 12 03:43:36 user nova-compute[72081]: node.session.cmds_max = 128
Apr 12 03:43:36 user nova-compute[72081]: node.session.queue_depth = 32
Apr 12 03:43:36 user nova-compute[72081]: node.session.nr_sessions = 1
Apr 12 03:43:36 user nova-compute[72081]: node.session.auth.authmethod = CHAP
Apr 12 03:43:36 user nova-compute[72081]: node.session.auth.username = fbzrSZhe
Apr 12 03:43:36 user nova-compute[72081]: node.session.auth.password = ***
Apr 12 03:43:36 user nova-compute[72081]: node.session.auth.username_in = <empty>
Apr 12 03:43:36 user nova-compute[72081]: node.session.auth.password_in = <empty>
Apr 12 03:43:36 user nova-compute[72081]: node.session.auth.chap_algs = MD5
Apr 12 03:43:36 user nova-compute[72081]: node.session.timeo.replacement_timeout = 120
Apr 12 03:43:36 user nova-compute[72081]: node.session.err_timeo.abort_timeout = 15
Apr 12 03:43:36 user nova-compute[72081]: node.session.err_timeo.lu_reset_timeout = 30
Apr 12 03:43:36 user nova-compute[72081]: node.session.err_timeo.tgt_reset_timeout = 30
Apr 12 03:43:36 user nova-compute[72081]: node.session.err_timeo.host_reset_timeout = 60
Apr 12 03:43:36 user nova-compute[72081]: node.session.iscsi.FastAbort = Yes
Apr 12 03:43:36 user nova-compute[72081]: node.session.iscsi.InitialR2T = No
Apr 12 03:43:36 user nova-compute[72081]: node.session.iscsi.ImmediateData = Yes
Apr 12 03:43:36 user nova-compute[72081]: node.session.iscsi.FirstBurstLength = 262144
Apr 12 03:43:36 user nova-compute[72081]: node.session.iscsi.MaxBurstLength = 16776192
Apr 12 03:43:36 user nova-compute[72081]: node.session.iscsi.DefaultTime2Retain = 0
Apr 12 03:43:36 user nova-compute[72081]: node.session.iscsi.DefaultTime2Wait = 2
Apr 12 03:43:36 user nova-compute[72081]: node.session.iscsi.MaxConnections = 1
Apr 12 03:43:36 user nova-compute[72081]: node.session.iscsi.MaxOutstandingR2T = 1
Apr 12 03:43:36 user nova-compute[72081]: node.session.iscsi.ERL = 0
Apr 12 03:43:36 user nova-compute[72081]: node.session.scan = manual
Apr 12 03:43:36 user nova-compute[72081]: node.session.reopen_max = 0
Apr 12 03:43:36 user nova-compute[72081]: node.conn[0].address = 172.16.0.220
Apr 12 03:43:36 user nova-compute[72081]: node.conn[0].port = 3260
Apr 12 03:43:36 user nova-compute[72081]: node.conn[0].startup = manual
Apr 12 03:43:36 user nova-compute[72081]: node.conn[0].tcp.window_size = 524288
Apr 12 03:43:36 user nova-compute[72081]: node.conn[0].tcp.type_of_service = 0
Apr 12 03:43:36 user nova-compute[72081]: node.conn[0].timeo.logout_timeout = 15
Apr 12 03:43:36 user nova-compute[72081]: node.conn[0].timeo.login_timeout = 15
Apr 12 03:43:36 user nova-compute[72081]: node.conn[0].timeo.auth_timeout = 45
Apr 12 03:43:36 user nova-compute[72081]: node.conn[0].timeo.noop_out_interval = 5
Apr 12 03:43:36 user nova-compute[72081]: node.conn[0].timeo.noop_out_timeout = 5
Apr 12 03:43:36 user nova-compute[72081]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0
Apr 12 03:43:36 user nova-compute[72081]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144
Apr 12 03:43:36 user nova-compute[72081]: node.conn[0].iscsi.HeaderDigest = None
Apr 12 03:43:36 user nova-compute[72081]: node.conn[0].iscsi.DataDigest = None
Apr 12 03:43:36 user nova-compute[72081]: node.conn[0].iscsi.IFMarker = No
Apr 12 03:43:36 user nova-compute[72081]: node.conn[0].iscsi.OFMarker = No
Apr 12 03:43:36 user nova-compute[72081]: # END RECORD
Apr 12 03:43:36 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[fbcf5904-fb92-4206-8967-3059b5322006]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e4a0bb02-a9a0-4ab1-950b-ea5bdd591520]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v fbzrSZhe {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v fbzrSZhe" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9b054e7e-ce61-4b64-b713-86ce5b2e13a0]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'fbzrSZhe'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[76660d22-c7ef-46f6-af27-575ee5df0280]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[947d7b3a-e415-4670-9520-808465de58dc]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:43:36 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:43:36 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:43:36 user nova-compute[72081]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:43:36 user nova-compute[72081]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:43:36 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:43:36 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:43:36 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:43:36 user nova-compute[72081]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:43:36 user nova-compute[72081]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:43:36 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.149s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] HCTL ('34', '0', '0', 0) found on session 28 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Scanning host 34 c: 0, t: 0, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[cae27593-0a10-4936-ab13-f9cd0cffc868]: (4, ('0 0 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Searching for a device in session 28 and hctl ('34', '0', '0', 0) yield: sdc {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'fbzrSZhe', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 0.172s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] <== connect_volume: return (173ms) {'type': 'block', 'scsi_wwn': '23037353334373734', 'path': '/dev/sdc'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23037353334373734', 'path': '/dev/sdc'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:41:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1156151605',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1156151605',id=30,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:41:58Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='6e1fa352a8c144a1aada7ce0d8818429',ramdisk_id='',reservation_id='r-r2n25ts0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-590302458',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:41:58Z,user_data=None,user_id='d99be569057f432484e4a60b46882bbd',uuid=614495f2-f108-4759-be60-bc6cc689c5b7,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "b9acaee6-a684-4827-bc91-1c6219782efe", "address": "fa:16:3e:d4:dc:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "vif_mac": "fa:16:3e:d4:dc:db"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb9acaee6-a6", "ovs_interfaceid": "b9acaee6-a684-4827-bc91-1c6219782efe", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converting VIF {"id": "b9acaee6-a684-4827-bc91-1c6219782efe", "address": "fa:16:3e:d4:dc:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "vif_mac": "fa:16:3e:d4:dc:db"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb9acaee6-a6", "ovs_interfaceid": "b9acaee6-a684-4827-bc91-1c6219782efe", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:d4:dc:db,bridge_name='br-int',has_traffic_filtering=True,id=b9acaee6-a684-4827-bc91-1c6219782efe,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb9acaee6-a6') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG nova.objects.instance [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lazy-loading 'pci_devices' on Instance uuid 614495f2-f108-4759-be60-bc6cc689c5b7 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:43:36 user nova-compute[72081]:   <uuid>614495f2-f108-4759-be60-bc6cc689c5b7</uuid>
Apr 12 03:43:36 user nova-compute[72081]:   <name>instance-0000001e</name>
Apr 12 03:43:36 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:43:36 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:43:36 user nova-compute[72081]:   <metadata>
Apr 12 03:43:36 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:43:36 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:43:36 user nova-compute[72081]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-1156151605</nova:name>
Apr 12 03:43:36 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:43:35</nova:creationTime>
Apr 12 03:43:36 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:43:36 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:43:36 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:43:36 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:43:36 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:43:36 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:43:36 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:43:36 user nova-compute[72081]:       <nova:owner>
Apr 12 03:43:36 user nova-compute[72081]:         <nova:user uuid="d99be569057f432484e4a60b46882bbd">tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member</nova:user>
Apr 12 03:43:36 user nova-compute[72081]:         <nova:project uuid="6e1fa352a8c144a1aada7ce0d8818429">tempest-ServerBootFromVolumeStableRescueTest-590302458</nova:project>
Apr 12 03:43:36 user nova-compute[72081]:       </nova:owner>
Apr 12 03:43:36 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:43:36 user nova-compute[72081]:       <nova:ports>
Apr 12 03:43:36 user nova-compute[72081]:         <nova:port uuid="b9acaee6-a684-4827-bc91-1c6219782efe">
Apr 12 03:43:36 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.8" ipVersion="4"/>
Apr 12 03:43:36 user nova-compute[72081]:         </nova:port>
Apr 12 03:43:36 user nova-compute[72081]:       </nova:ports>
Apr 12 03:43:36 user nova-compute[72081]:     </nova:instance>
Apr 12 03:43:36 user nova-compute[72081]:   </metadata>
Apr 12 03:43:36 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:43:36 user nova-compute[72081]:     <system>
Apr 12 03:43:36 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:43:36 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:43:36 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:43:36 user nova-compute[72081]:       <entry name="serial">614495f2-f108-4759-be60-bc6cc689c5b7</entry>
Apr 12 03:43:36 user nova-compute[72081]:       <entry name="uuid">614495f2-f108-4759-be60-bc6cc689c5b7</entry>
Apr 12 03:43:36 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:43:36 user nova-compute[72081]:     </system>
Apr 12 03:43:36 user nova-compute[72081]:   </sysinfo>
Apr 12 03:43:36 user nova-compute[72081]:   <os>
Apr 12 03:43:36 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:43:36 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:43:36 user nova-compute[72081]:   </os>
Apr 12 03:43:36 user nova-compute[72081]:   <features>
Apr 12 03:43:36 user nova-compute[72081]:     <acpi/>
Apr 12 03:43:36 user nova-compute[72081]:     <apic/>
Apr 12 03:43:36 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:43:36 user nova-compute[72081]:   </features>
Apr 12 03:43:36 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:43:36 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:43:36 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:43:36 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:43:36 user nova-compute[72081]:   </clock>
Apr 12 03:43:36 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:43:36 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:43:36 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:43:36 user nova-compute[72081]:   </cpu>
Apr 12 03:43:36 user nova-compute[72081]:   <devices>
Apr 12 03:43:36 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:43:36 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:43:36 user nova-compute[72081]:       <source dev="/dev/sdc"/>
Apr 12 03:43:36 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:43:36 user nova-compute[72081]:       <serial>54a9604a-4853-414c-850e-51745d6fcd0e</serial>
Apr 12 03:43:36 user nova-compute[72081]:     </disk>
Apr 12 03:43:36 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:43:36 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:43:36 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/614495f2-f108-4759-be60-bc6cc689c5b7/disk.rescue"/>
Apr 12 03:43:36 user nova-compute[72081]:       <target dev="vdb" bus="virtio"/>
Apr 12 03:43:36 user nova-compute[72081]:       <boot order="1"/>
Apr 12 03:43:36 user nova-compute[72081]:     </disk>
Apr 12 03:43:36 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:43:36 user nova-compute[72081]:       <mac address="fa:16:3e:d4:dc:db"/>
Apr 12 03:43:36 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:43:36 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:43:36 user nova-compute[72081]:       <target dev="tapb9acaee6-a6"/>
Apr 12 03:43:36 user nova-compute[72081]:     </interface>
Apr 12 03:43:36 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:43:36 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/614495f2-f108-4759-be60-bc6cc689c5b7/console.log" append="off"/>
Apr 12 03:43:36 user nova-compute[72081]:     </serial>
Apr 12 03:43:36 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:43:36 user nova-compute[72081]:     <video>
Apr 12 03:43:36 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:43:36 user nova-compute[72081]:     </video>
Apr 12 03:43:36 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:43:36 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:43:36 user nova-compute[72081]:     </rng>
Apr 12 03:43:36 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:43:36 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:43:36 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:43:36 user nova-compute[72081]:     </memballoon>
Apr 12 03:43:36 user nova-compute[72081]:   </devices>
Apr 12 03:43:36 user nova-compute[72081]: </domain>
Apr 12 03:43:36 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:43:36 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Instance destroyed successfully.
Apr 12 03:43:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] No VIF found with MAC fa:16:3e:d4:dc:db, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-73e6b8c2-a89d-44d4-9802-df9a77279d3a req-6594fd19-d050-422f-91cf-89af0e39dd92 service nova] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Received event network-vif-plugged-e210f496-084e-443f-9c7f-684661441f6c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:43:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-73e6b8c2-a89d-44d4-9802-df9a77279d3a req-6594fd19-d050-422f-91cf-89af0e39dd92 service nova] Acquiring lock "93e45116-bb5c-40ca-9fc5-1c5c40ff687e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-73e6b8c2-a89d-44d4-9802-df9a77279d3a req-6594fd19-d050-422f-91cf-89af0e39dd92 service nova] Lock "93e45116-bb5c-40ca-9fc5-1c5c40ff687e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-73e6b8c2-a89d-44d4-9802-df9a77279d3a req-6594fd19-d050-422f-91cf-89af0e39dd92 service nova] Lock "93e45116-bb5c-40ca-9fc5-1c5c40ff687e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-73e6b8c2-a89d-44d4-9802-df9a77279d3a req-6594fd19-d050-422f-91cf-89af0e39dd92 service nova] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] No waiting events found dispatching network-vif-plugged-e210f496-084e-443f-9c7f-684661441f6c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:43:37 user nova-compute[72081]: WARNING nova.compute.manager [req-73e6b8c2-a89d-44d4-9802-df9a77279d3a req-6594fd19-d050-422f-91cf-89af0e39dd92 service nova] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Received unexpected event network-vif-plugged-e210f496-084e-443f-9c7f-684661441f6c for instance with vm_state building and task_state spawning.
Apr 12 03:43:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-334804da-0c67-45fd-a32c-d4aa7cd6c2b5 req-b9a61087-c064-4657-8b61-c663900814f5 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received event network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:43:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-334804da-0c67-45fd-a32c-d4aa7cd6c2b5 req-b9a61087-c064-4657-8b61-c663900814f5 service nova] Acquiring lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-334804da-0c67-45fd-a32c-d4aa7cd6c2b5 req-b9a61087-c064-4657-8b61-c663900814f5 service nova] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-334804da-0c67-45fd-a32c-d4aa7cd6c2b5 req-b9a61087-c064-4657-8b61-c663900814f5 service nova] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-334804da-0c67-45fd-a32c-d4aa7cd6c2b5 req-b9a61087-c064-4657-8b61-c663900814f5 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] No waiting events found dispatching network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:43:38 user nova-compute[72081]: WARNING nova.compute.manager [req-334804da-0c67-45fd-a32c-d4aa7cd6c2b5 req-b9a61087-c064-4657-8b61-c663900814f5 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received unexpected event network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe for instance with vm_state active and task_state rescuing.
Apr 12 03:43:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:39 user nova-compute[72081]: DEBUG nova.compute.manager [req-302424f0-57d5-46a8-9fe9-661707ff8bb7 req-3dcca29b-f190-4a34-89b8-7e0428ad20fd service nova] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Received event network-vif-plugged-e210f496-084e-443f-9c7f-684661441f6c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:43:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-302424f0-57d5-46a8-9fe9-661707ff8bb7 req-3dcca29b-f190-4a34-89b8-7e0428ad20fd service nova] Acquiring lock "93e45116-bb5c-40ca-9fc5-1c5c40ff687e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-302424f0-57d5-46a8-9fe9-661707ff8bb7 req-3dcca29b-f190-4a34-89b8-7e0428ad20fd service nova] Lock "93e45116-bb5c-40ca-9fc5-1c5c40ff687e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-302424f0-57d5-46a8-9fe9-661707ff8bb7 req-3dcca29b-f190-4a34-89b8-7e0428ad20fd service nova] Lock "93e45116-bb5c-40ca-9fc5-1c5c40ff687e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:39 user nova-compute[72081]: DEBUG nova.compute.manager [req-302424f0-57d5-46a8-9fe9-661707ff8bb7 req-3dcca29b-f190-4a34-89b8-7e0428ad20fd service nova] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] No waiting events found dispatching network-vif-plugged-e210f496-084e-443f-9c7f-684661441f6c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:43:39 user nova-compute[72081]: WARNING nova.compute.manager [req-302424f0-57d5-46a8-9fe9-661707ff8bb7 req-3dcca29b-f190-4a34-89b8-7e0428ad20fd service nova] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Received unexpected event network-vif-plugged-e210f496-084e-443f-9c7f-684661441f6c for instance with vm_state building and task_state spawning.
Apr 12 03:43:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.compute.manager [req-3d02715e-aca5-47cb-8013-d3f61c7f2842 req-54d155aa-52c2-4342-aa66-d58edd6a96fd service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received event network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:43:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3d02715e-aca5-47cb-8013-d3f61c7f2842 req-54d155aa-52c2-4342-aa66-d58edd6a96fd service nova] Acquiring lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3d02715e-aca5-47cb-8013-d3f61c7f2842 req-54d155aa-52c2-4342-aa66-d58edd6a96fd service nova] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3d02715e-aca5-47cb-8013-d3f61c7f2842 req-54d155aa-52c2-4342-aa66-d58edd6a96fd service nova] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.compute.manager [req-3d02715e-aca5-47cb-8013-d3f61c7f2842 req-54d155aa-52c2-4342-aa66-d58edd6a96fd service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] No waiting events found dispatching network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:43:40 user nova-compute[72081]: WARNING nova.compute.manager [req-3d02715e-aca5-47cb-8013-d3f61c7f2842 req-54d155aa-52c2-4342-aa66-d58edd6a96fd service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received unexpected event network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe for instance with vm_state active and task_state rescuing.
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681271005.4184172, f88ecf99-d158-4f3c-8a87-6c0322fdc8ab => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:43:40 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] VM Stopped (Lifecycle Event)
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-7748f607-971a-488a-822d-a2bcfd3c6f6f None None] [instance: f88ecf99-d158-4f3c-8a87-6c0322fdc8ab] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271020.4518528, 93e45116-bb5c-40ca-9fc5-1c5c40ff687e => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:43:40 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] VM Resumed (Lifecycle Event)
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:43:40 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Instance spawned successfully.
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-a0ed1f44-19dd-4fe9-9f36-3a7f5dd53120 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:43:40 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271020.4592667, 93e45116-bb5c-40ca-9fc5-1c5c40ff687e => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:43:40 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] VM Started (Lifecycle Event)
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:43:40 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Removed pending event for 614495f2-f108-4759-be60-bc6cc689c5b7 due to event {{(pid=72081) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271020.4670095, 614495f2-f108-4759-be60-bc6cc689c5b7 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:43:40 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] VM Resumed (Lifecycle Event)
Apr 12 03:43:40 user nova-compute[72081]: INFO nova.compute.manager [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Took 7.50 seconds to spawn the instance on the hypervisor.
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: rescued, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271020.4682674, 614495f2-f108-4759-be60-bc6cc689c5b7 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:43:40 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] VM Started (Lifecycle Event)
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:43:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Synchronizing instance power state after lifecycle event "Started"; current vm_state: rescued, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:43:40 user nova-compute[72081]: INFO nova.compute.manager [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Took 12.84 seconds to build instance.
Apr 12 03:43:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4f9c671c-190f-4d19-a535-2db58727e545 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "93e45116-bb5c-40ca-9fc5-1c5c40ff687e" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 12.927s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:45 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-bf68c7a4-77f4-48c3-9d3e-9508cdec2a98 req-84247dfa-0fc2-4ff6-972d-4be8b47dca2d service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received event network-changed-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:43:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-bf68c7a4-77f4-48c3-9d3e-9508cdec2a98 req-84247dfa-0fc2-4ff6-972d-4be8b47dca2d service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Refreshing instance network info cache due to event network-changed-e16af111-f1d4-4f8c-a67a-c80f546c37b1. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:43:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bf68c7a4-77f4-48c3-9d3e-9508cdec2a98 req-84247dfa-0fc2-4ff6-972d-4be8b47dca2d service nova] Acquiring lock "refresh_cache-606425bb-8968-41f9-b3a5-80b6fd188a67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:43:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bf68c7a4-77f4-48c3-9d3e-9508cdec2a98 req-84247dfa-0fc2-4ff6-972d-4be8b47dca2d service nova] Acquired lock "refresh_cache-606425bb-8968-41f9-b3a5-80b6fd188a67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:43:53 user nova-compute[72081]: DEBUG nova.network.neutron [req-bf68c7a4-77f4-48c3-9d3e-9508cdec2a98 req-84247dfa-0fc2-4ff6-972d-4be8b47dca2d service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Refreshing network info cache for port e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:43:54 user nova-compute[72081]: DEBUG nova.network.neutron [req-bf68c7a4-77f4-48c3-9d3e-9508cdec2a98 req-84247dfa-0fc2-4ff6-972d-4be8b47dca2d service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Updated VIF entry in instance network info cache for port e16af111-f1d4-4f8c-a67a-c80f546c37b1. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:43:54 user nova-compute[72081]: DEBUG nova.network.neutron [req-bf68c7a4-77f4-48c3-9d3e-9508cdec2a98 req-84247dfa-0fc2-4ff6-972d-4be8b47dca2d service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Updating instance_info_cache with network_info: [{"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.205", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:43:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bf68c7a4-77f4-48c3-9d3e-9508cdec2a98 req-84247dfa-0fc2-4ff6-972d-4be8b47dca2d service nova] Releasing lock "refresh_cache-606425bb-8968-41f9-b3a5-80b6fd188a67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:43:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:55 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'flavor' on Instance uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:43:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.056s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:55 user nova-compute[72081]: DEBUG nova.network.neutron [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Port 18f9f68c-eadd-4582-8f16-b88a13496852 binding to destination host user is already ACTIVE {{(pid=72081) migrate_instance_start /opt/stack/nova/nova/network/neutron.py:3161}}
Apr 12 03:43:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:43:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquired lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:43:55 user nova-compute[72081]: DEBUG nova.network.neutron [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:43:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:55 user nova-compute[72081]: INFO nova.compute.manager [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Attaching volume 188c90f2-90ef-4909-8917-56808fd9a068 to /dev/vdb
Apr 12 03:43:55 user nova-compute[72081]: DEBUG os_brick.utils [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:43:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7b83a133-2411-41e1-8e7d-ce6ce95b6cb4]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f2a71a24-4da8-4915-b7ea-e531737cb402]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[86299b1a-d5d4-424f-b872-82d6320acb0d]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb85910f-28e2-4e87-af79-97332f90bcc2]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:43:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:43:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:43:56 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:43:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ed9d613-f9ee-499b-8866-fdace500749b]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:43:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:43:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:43:56 user nova-compute[72081]: DEBUG os_brick.utils [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] <== get_connector_properties: return (143ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:43:56 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Updating existing volume attachment record: b6f0bcbc-0459-4eea-bd06-9911027035a9 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:43:56 user nova-compute[72081]: DEBUG nova.network.neutron [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Updating instance_info_cache with network_info: [{"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:43:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Releasing lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:43:56 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Creating tmpfile /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/tmp78crq4go to verify with other compute node that the instance is on the same shared storage. {{(pid=72081) check_instance_shared_storage_local /opt/stack/nova/nova/virt/libvirt/driver.py:9595}}
Apr 12 03:43:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:56 user nova-compute[72081]: DEBUG nova.compute.manager [req-37536abe-1398-48ed-b9c9-ead82a2cec5f req-6427c51d-c3f8-4f99-8418-17bfe3ca6023 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received event network-vif-unplugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:43:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-37536abe-1398-48ed-b9c9-ead82a2cec5f req-6427c51d-c3f8-4f99-8418-17bfe3ca6023 service nova] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-37536abe-1398-48ed-b9c9-ead82a2cec5f req-6427c51d-c3f8-4f99-8418-17bfe3ca6023 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-37536abe-1398-48ed-b9c9-ead82a2cec5f req-6427c51d-c3f8-4f99-8418-17bfe3ca6023 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:56 user nova-compute[72081]: DEBUG nova.compute.manager [req-37536abe-1398-48ed-b9c9-ead82a2cec5f req-6427c51d-c3f8-4f99-8418-17bfe3ca6023 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] No waiting events found dispatching network-vif-unplugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:43:56 user nova-compute[72081]: WARNING nova.compute.manager [req-37536abe-1398-48ed-b9c9-ead82a2cec5f req-6427c51d-c3f8-4f99-8418-17bfe3ca6023 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received unexpected event network-vif-unplugged-18f9f68c-eadd-4582-8f16-b88a13496852 for instance with vm_state resized and task_state resize_reverting.
Apr 12 03:43:57 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Instance destroyed successfully.
Apr 12 03:43:57 user nova-compute[72081]: DEBUG nova.objects.instance [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'resources' on Instance uuid 48977eb2-ce43-4577-a1cb-340e35703dbd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-12T03:33:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-2050314981',display_name='tempest-ServerActionsTestJSON-server-1847743592',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(12),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-2050314981',id=14,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=12,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKmrI9wy1CUIwhlKuXvzk0mD7Pt5nZFvI1CsqS0do6Lk76JQOvGl5lvx9rkz7UAM9WGocGXyWSOUOuKTtYZRD2HqKUbTN4VPOoliE1yuu+s0lx7vuZu45Mj7wDb2S8Ksyw==',key_name='tempest-keypair-364276420',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:42:23Z,launched_on='user',locked=False,locked_by=None,memory_mb=192,metadata={rebuild='server'},migration_context=<?>,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=Flavor(11),os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='815b65aa5ad9442db0a087c646f96bbd',ramdisk_id='',reservation_id='r-usz98205',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1898059859',owner_user_name='tempest-ServerActionsTestJSON-1898059859-project-member'},tags=<?>,task_state='resize_reverting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:42:24Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a7f422f1a05a4a92ba387e424850b360',uuid=48977eb2-ce43-4577-a1cb-340e35703dbd,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='resized') vif={"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converting VIF {"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG os_vif [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap18f9f68c-ea, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:43:57 user nova-compute[72081]: INFO os_vif [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea')
Apr 12 03:43:57 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YDhqrwbV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdg': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:43:57 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:43:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[911aa77d-6884-474e-b7da-f65f1872f339]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:43:57 user nova-compute[72081]: No targets found.
Apr 12 03:43:57 user nova-compute[72081]: iSNS:
Apr 12 03:43:57 user nova-compute[72081]: No targets found.
Apr 12 03:43:57 user nova-compute[72081]: STATIC:
Apr 12 03:43:57 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27
Apr 12 03:43:57 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:43:57 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:43:57 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:43:57 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:43:57 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:43:57 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e
Apr 12 03:43:57 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:43:57 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:43:57 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3
Apr 12 03:43:57 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:43:57 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:43:57 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742
Apr 12 03:43:57 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:43:57 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:43:57 user nova-compute[72081]: FIRMWARE:
Apr 12 03:43:57 user nova-compute[72081]: No targets found.
Apr 12 03:43:57 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:43:57 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:43:57 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1699f0aa-5d86-40f2-afcb-b784ebc883d5]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[71329799-c884-4d69-9c6c-5c44cdfef3c6]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:43:57 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:43:57 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:43:57 user nova-compute[72081]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:43:57 user nova-compute[72081]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:43:57 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:43:57 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:43:57 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:43:57 user nova-compute[72081]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:43:57 user nova-compute[72081]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:43:57 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fb403b130>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742'): ({'sdg'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Removing single pathed devices sdg {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7babaa37-4772-4a06-8752-4313b48d5d5b]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdg {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdg" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3ce093b6-453f-415e-aeec-fe65df7d7da5]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Flushing IO for device /dev/sdg {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdg {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdg" returned: 0 in 0.013s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ea9f4d38-3f39-4fd5-befb-eafe6a5173cf]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Remove SCSI device /dev/sdg with /sys/block/sdg/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdg/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdg/device/delete" returned: 0 in 0.041s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d2eec334-5030-47ac-b07a-8cdcf67853f0]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Checking to see if SCSI volumes sdg have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] SCSI volumes sdg have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f586ab3c-4152-439a-b163-6f0a6c352f66]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --logout" returned: 0 in 0.055s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e1e830ed-6753-4250-9382-72c1c2625141]: (4, ('Logging out of session [sid: 31, target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742, portal: 172.16.0.220,3260]\nLogout of [sid: 31, target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 31, target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742, portal: 172.16.0.220,3260]
Apr 12 03:43:57 user nova-compute[72081]: Logout of [sid: 31, target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742, portal: 172.16.0.220,3260] successful.
Apr 12 03:43:57 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f355bcc4-3473-42ad-8dd5-3e79c218c64d]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.302s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] <== disconnect_volume: return (303ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:43:57 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:43:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:43:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_dest" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_dest" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:58 user nova-compute[72081]: DEBUG nova.objects.instance [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'migration_context' on Instance uuid 48977eb2-ce43-4577-a1cb-340e35703dbd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:43:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-5a08fa23-d8c0-40d9-9800-3f4c4f79fd9e req-3baeb5ba-100d-4903-a36e-5a1cc0b4200f service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:43:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5a08fa23-d8c0-40d9-9800-3f4c4f79fd9e req-3baeb5ba-100d-4903-a36e-5a1cc0b4200f service nova] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:43:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5a08fa23-d8c0-40d9-9800-3f4c4f79fd9e req-3baeb5ba-100d-4903-a36e-5a1cc0b4200f service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:43:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5a08fa23-d8c0-40d9-9800-3f4c4f79fd9e req-3baeb5ba-100d-4903-a36e-5a1cc0b4200f service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-5a08fa23-d8c0-40d9-9800-3f4c4f79fd9e req-3baeb5ba-100d-4903-a36e-5a1cc0b4200f service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] No waiting events found dispatching network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:43:58 user nova-compute[72081]: WARNING nova.compute.manager [req-5a08fa23-d8c0-40d9-9800-3f4c4f79fd9e req-3baeb5ba-100d-4903-a36e-5a1cc0b4200f service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received unexpected event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 for instance with vm_state resized and task_state resize_reverting.
Apr 12 03:43:58 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:43:58 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:43:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_dest" :: held 0.419s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:43:59 user nova-compute[72081]: INFO nova.compute.manager [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Swapping old allocation on dict_keys(['2203ce45-27b6-4606-b33a-0f78fa74c5d6']) held by migration 35efd868-6d17-47a5-bc6f-dc21edd02c98 for instance
Apr 12 03:43:59 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Overwriting current allocation {'allocations': {'2203ce45-27b6-4606-b33a-0f78fa74c5d6': {'resources': {'VCPU': 1, 'MEMORY_MB': 192, 'DISK_GB': 1}, 'generation': 38}}, 'project_id': '815b65aa5ad9442db0a087c646f96bbd', 'user_id': 'a7f422f1a05a4a92ba387e424850b360', 'consumer_generation': 1} on consumer 48977eb2-ce43-4577-a1cb-340e35703dbd {{(pid=72081) move_allocations /opt/stack/nova/nova/scheduler/client/report.py:2018}}
Apr 12 03:43:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:43:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquired lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:43:59 user nova-compute[72081]: DEBUG nova.network.neutron [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:43:59 user nova-compute[72081]: DEBUG nova.network.neutron [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Updating instance_info_cache with network_info: [{"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:43:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Releasing lock "refresh_cache-48977eb2-ce43-4577-a1cb-340e35703dbd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:43:59 user nova-compute[72081]: DEBUG os_brick.utils [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:43:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8297775a-6db6-4efd-a81a-53475725c48c]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[effaf250-a4da-48cf-886f-31874105c956]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:43:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2c88ef6b-b458-49f5-a4ae-92051141b81f]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f291d517-9961-41a2-97e6-02201aae5783]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:43:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:43:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:43:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:43:59 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:43:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[383ebe21-000c-48f8-a8bd-9b4c684f12c8]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:43:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:43:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:43:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:43:59 user nova-compute[72081]: DEBUG os_brick.utils [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] <== get_connector_properties: return (123ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'VoqWQaSf', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:44:00 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:44:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:44:00 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:44:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:44:00 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[24f5f3c1-06d5-4c42-9aa8-4218aae2a8d7]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:44:00 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b6034874-7244-4177-9684-8ca77803c485]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068] added
Apr 12 03:44:00 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4d5f7f54-8e76-41e9-896a-dc2918a5cd5e]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4b415d64-fd4e-4708-91e6-f9d587ae3183]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v VoqWQaSf {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v VoqWQaSf" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3c73756e-f209-4a78-b245-513c3aad35ea]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'VoqWQaSf'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.029s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[81ffe48b-093f-4e09-b562-6b1f5467d6ff]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d794734d-53c2-4708-b3d8-7aa372f40523]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:44:00 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:44:00 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:44:00 user nova-compute[72081]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:44:00 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:44:00 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:44:00 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:44:00 user nova-compute[72081]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:44:00 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --login" returned: 0 in 0.073s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[66a17a63-ea83-4a86-94c0-306e6ef8685a]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068, portal: 172.16.0.220,3260]
Apr 12 03:44:00 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068, portal: 172.16.0.220,3260] successful.
Apr 12 03:44:00 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[973b2974-2f70-419b-8611-866ce4b03848]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bcf3d71d-a554-4a7d-96b5-26a241537280]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:44:00 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:44:00 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:44:00 user nova-compute[72081]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:44:00 user nova-compute[72081]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:44:00 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:44:00 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:44:00 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:44:00 user nova-compute[72081]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:44:00 user nova-compute[72081]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:44:00 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.320s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] HCTL ('36', '-', '-', 0) found on session 33 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[acd8f283-5d6a-4eab-8af7-a434b5e05c2f]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Searching for a device in session 33 and hctl ['36', '*', '*', 0] yield: sde {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:44:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'VoqWQaSf', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:44:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.363s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:44:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] <== connect_volume: return (1366ms) {'type': 'block', 'scsi_wwn': '23539616538316433', 'path': '/dev/sde'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:44:01 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23539616538316433', 'path': '/dev/sde'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:44:01 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'flavor' on Instance uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:44:01 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] attach device xml: <disk type="block" device="disk">
Apr 12 03:44:01 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:44:01 user nova-compute[72081]:   <source dev="/dev/sde"/>
Apr 12 03:44:01 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:44:01 user nova-compute[72081]:   <serial>188c90f2-90ef-4909-8917-56808fd9a068</serial>
Apr 12 03:44:01 user nova-compute[72081]: </disk>
Apr 12 03:44:01 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 03:44:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:02 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:44:02 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:44:02 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] No VIF found with MAC fa:16:3e:85:88:9b, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:44:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4cff64a2-b726-4aea-8cf3-0f3e03359101 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.546s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:44:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2ee38eec-f7fd-4b9d-afcd-b2b93a73559d tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67" by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:44:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2ee38eec-f7fd-4b9d-afcd-b2b93a73559d tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67" acquired by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:44:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-2ee38eec-f7fd-4b9d-afcd-b2b93a73559d tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:44:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-2ee38eec-f7fd-4b9d-afcd-b2b93a73559d tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Stopping instance; current vm_state: active, current task_state: powering-off, current DB power_state: 1, current VM power_state: 1 {{(pid=72081) do_stop_instance /opt/stack/nova/nova/compute/manager.py:3323}}
Apr 12 03:44:03 user nova-compute[72081]: DEBUG nova.objects.instance [None req-2ee38eec-f7fd-4b9d-afcd-b2b93a73559d tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'flavor' on Instance uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:44:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-9af32e15-52d8-4b90-b87d-3b75ece8cd53 req-8eb5954f-3e3a-43a9-800f-10bb48e30ca2 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received event network-vif-unplugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9af32e15-52d8-4b90-b87d-3b75ece8cd53 req-8eb5954f-3e3a-43a9-800f-10bb48e30ca2 service nova] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9af32e15-52d8-4b90-b87d-3b75ece8cd53 req-8eb5954f-3e3a-43a9-800f-10bb48e30ca2 service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9af32e15-52d8-4b90-b87d-3b75ece8cd53 req-8eb5954f-3e3a-43a9-800f-10bb48e30ca2 service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-9af32e15-52d8-4b90-b87d-3b75ece8cd53 req-8eb5954f-3e3a-43a9-800f-10bb48e30ca2 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] No waiting events found dispatching network-vif-unplugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:44:04 user nova-compute[72081]: WARNING nova.compute.manager [req-9af32e15-52d8-4b90-b87d-3b75ece8cd53 req-8eb5954f-3e3a-43a9-800f-10bb48e30ca2 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received unexpected event network-vif-unplugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 for instance with vm_state active and task_state powering-off.
Apr 12 03:44:04 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Instance destroyed successfully.
Apr 12 03:44:04 user nova-compute[72081]: DEBUG nova.compute.manager [None req-2ee38eec-f7fd-4b9d-afcd-b2b93a73559d tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2ee38eec-f7fd-4b9d-afcd-b2b93a73559d tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67" "released" by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" :: held 0.924s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Starting finish_revert_migration {{(pid=72081) finish_revert_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11792}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Start _get_guest_xml network_info=[{"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vdb', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YDhqrwbV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'attaching', 'instance': '48977eb2-ce43-4577-a1cb-340e35703dbd', 'attached_at': '2023-04-12T03:44:04.000000', 'detached_at': '', 'volume_id': '8278d82d-cc19-4660-b4f1-b13d92e7d742', 'serial': '8278d82d-cc19-4660-b4f1-b13d92e7d742'}, 'disk_bus': 'virtio', 'delete_on_termination': False, 'boot_index': None, 'guest_format': None, 'device_type': 'disk', 'attachment_id': '9fb3a409-d73d-476a-aceb-9d96a313cfbf', 'volume_type': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:44:04 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:44:04 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:44:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG nova.objects.instance [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'vcpu_model' on Instance uuid 48977eb2-ce43-4577-a1cb-340e35703dbd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YDhqrwbV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:44:04 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:44:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:44:04 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:44:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:44:04 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1ed37985-c627-4717-bc97-0e9b8ba04cca]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:44:04 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[56815331-1ea9-4c60-8462-312f0ca950da]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742] added
Apr 12 03:44:04 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f9489389-68fd-4502-abf7-ecae7e39a7e3]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[baa5257a-0c7b-4be3-929e-9381502d22ce]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YDhqrwbV {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YDhqrwbV" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d9ecf627-7495-4bf6-9b78-d73ae43da205]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'YDhqrwbV'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6f3321bf-15a8-4d5c-8c7b-14ba96f57058]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6b80532e-d1a4-4477-8ebe-f1f4eb1b48bb]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:44:04 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:44:04 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:44:04 user nova-compute[72081]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:44:04 user nova-compute[72081]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:44:04 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:44:04 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:44:04 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:44:04 user nova-compute[72081]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:44:04 user nova-compute[72081]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:44:04 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --login" returned: 0 in 0.071s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bdab0503-1fc2-4a75-9be9-175687716b91]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742, portal: 172.16.0.220,3260]
Apr 12 03:44:04 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742, portal: 172.16.0.220,3260] successful.
Apr 12 03:44:04 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bc8c3d12-612f-49d1-8b89-bba6364b2d9b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[fb3f05d2-6573-4ab5-ab57-94bb7a7651a5]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:44:04 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:44:04 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:44:04 user nova-compute[72081]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:44:04 user nova-compute[72081]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:44:04 user nova-compute[72081]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:44:04 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:44:04 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:44:04 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:44:04 user nova-compute[72081]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:44:04 user nova-compute[72081]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:44:04 user nova-compute[72081]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:44:04 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.308s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] HCTL ('38', '-', '-', 0) found on session 34 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Scanning host 38 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host38/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host38/scan" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[39a457b6-7d13-4328-b7a6-e3659e13ed42]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Searching for a device in session 34 and hctl ['38', '*', '*', 0] yield: sdg {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:44:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Connected to sdg using {'target_iqn': 'iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YDhqrwbV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:44:05 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:44:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.342s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:44:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] <== connect_volume: return (1344ms) {'type': 'block', 'scsi_wwn': '26437356130623139', 'path': '/dev/sdg'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:44:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26437356130623139', 'path': '/dev/sdg'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:44:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-12T03:33:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-2050314981',display_name='tempest-ServerActionsTestJSON-server-1847743592',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-2050314981',id=14,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKmrI9wy1CUIwhlKuXvzk0mD7Pt5nZFvI1CsqS0do6Lk76JQOvGl5lvx9rkz7UAM9WGocGXyWSOUOuKTtYZRD2HqKUbTN4VPOoliE1yuu+s0lx7vuZu45Mj7wDb2S8Ksyw==',key_name='tempest-keypair-364276420',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:42:23Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={rebuild='server'},migration_context=MigrationContext,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=Flavor(11),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='815b65aa5ad9442db0a087c646f96bbd',ramdisk_id='',reservation_id='r-usz98205',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1898059859',owner_user_name='tempest-ServerActionsTestJSON-1898059859-project-member'},tags=<?>,task_state='resize_reverting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:43:55Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a7f422f1a05a4a92ba387e424850b360',uuid=48977eb2-ce43-4577-a1cb-340e35703dbd,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='resized') vif={"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:44:06 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converting VIF {"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:44:06 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:44:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:44:06 user nova-compute[72081]:   <uuid>48977eb2-ce43-4577-a1cb-340e35703dbd</uuid>
Apr 12 03:44:06 user nova-compute[72081]:   <name>instance-0000000e</name>
Apr 12 03:44:06 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:44:06 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:44:06 user nova-compute[72081]:   <metadata>
Apr 12 03:44:06 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:44:06 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:44:06 user nova-compute[72081]:       <nova:name>tempest-ServerActionsTestJSON-server-1847743592</nova:name>
Apr 12 03:44:06 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:44:04</nova:creationTime>
Apr 12 03:44:06 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:44:06 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:44:06 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:44:06 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:44:06 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:44:06 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:44:06 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:44:06 user nova-compute[72081]:       <nova:owner>
Apr 12 03:44:06 user nova-compute[72081]:         <nova:user uuid="a7f422f1a05a4a92ba387e424850b360">tempest-ServerActionsTestJSON-1898059859-project-member</nova:user>
Apr 12 03:44:06 user nova-compute[72081]:         <nova:project uuid="815b65aa5ad9442db0a087c646f96bbd">tempest-ServerActionsTestJSON-1898059859</nova:project>
Apr 12 03:44:06 user nova-compute[72081]:       </nova:owner>
Apr 12 03:44:06 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:44:06 user nova-compute[72081]:       <nova:ports>
Apr 12 03:44:06 user nova-compute[72081]:         <nova:port uuid="18f9f68c-eadd-4582-8f16-b88a13496852">
Apr 12 03:44:06 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 12 03:44:06 user nova-compute[72081]:         </nova:port>
Apr 12 03:44:06 user nova-compute[72081]:       </nova:ports>
Apr 12 03:44:06 user nova-compute[72081]:     </nova:instance>
Apr 12 03:44:06 user nova-compute[72081]:   </metadata>
Apr 12 03:44:06 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:44:06 user nova-compute[72081]:     <system>
Apr 12 03:44:06 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:44:06 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:44:06 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:44:06 user nova-compute[72081]:       <entry name="serial">48977eb2-ce43-4577-a1cb-340e35703dbd</entry>
Apr 12 03:44:06 user nova-compute[72081]:       <entry name="uuid">48977eb2-ce43-4577-a1cb-340e35703dbd</entry>
Apr 12 03:44:06 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:44:06 user nova-compute[72081]:     </system>
Apr 12 03:44:06 user nova-compute[72081]:   </sysinfo>
Apr 12 03:44:06 user nova-compute[72081]:   <os>
Apr 12 03:44:06 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:44:06 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:44:06 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:44:06 user nova-compute[72081]:   </os>
Apr 12 03:44:06 user nova-compute[72081]:   <features>
Apr 12 03:44:06 user nova-compute[72081]:     <acpi/>
Apr 12 03:44:06 user nova-compute[72081]:     <apic/>
Apr 12 03:44:06 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:44:06 user nova-compute[72081]:   </features>
Apr 12 03:44:06 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:44:06 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:44:06 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:44:06 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:44:06 user nova-compute[72081]:   </clock>
Apr 12 03:44:06 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:44:06 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:44:06 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:44:06 user nova-compute[72081]:   </cpu>
Apr 12 03:44:06 user nova-compute[72081]:   <devices>
Apr 12 03:44:06 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:44:06 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:44:06 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk"/>
Apr 12 03:44:06 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:44:06 user nova-compute[72081]:     </disk>
Apr 12 03:44:06 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:44:06 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:44:06 user nova-compute[72081]:       <source dev="/dev/sdg"/>
Apr 12 03:44:06 user nova-compute[72081]:       <target dev="vdb" bus="virtio"/>
Apr 12 03:44:06 user nova-compute[72081]:       <serial>8278d82d-cc19-4660-b4f1-b13d92e7d742</serial>
Apr 12 03:44:06 user nova-compute[72081]:     </disk>
Apr 12 03:44:06 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:44:06 user nova-compute[72081]:       <mac address="fa:16:3e:b3:c5:ca"/>
Apr 12 03:44:06 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:44:06 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:44:06 user nova-compute[72081]:       <target dev="tap18f9f68c-ea"/>
Apr 12 03:44:06 user nova-compute[72081]:     </interface>
Apr 12 03:44:06 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:44:06 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/console.log" append="off"/>
Apr 12 03:44:06 user nova-compute[72081]:     </serial>
Apr 12 03:44:06 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:44:06 user nova-compute[72081]:     <video>
Apr 12 03:44:06 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:44:06 user nova-compute[72081]:     </video>
Apr 12 03:44:06 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:44:06 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:44:06 user nova-compute[72081]:     </rng>
Apr 12 03:44:06 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:44:06 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:44:06 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:44:06 user nova-compute[72081]:     </memballoon>
Apr 12 03:44:06 user nova-compute[72081]:   </devices>
Apr 12 03:44:06 user nova-compute[72081]: </domain>
Apr 12 03:44:06 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:44:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-12T03:33:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-2050314981',display_name='tempest-ServerActionsTestJSON-server-1847743592',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-2050314981',id=14,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKmrI9wy1CUIwhlKuXvzk0mD7Pt5nZFvI1CsqS0do6Lk76JQOvGl5lvx9rkz7UAM9WGocGXyWSOUOuKTtYZRD2HqKUbTN4VPOoliE1yuu+s0lx7vuZu45Mj7wDb2S8Ksyw==',key_name='tempest-keypair-364276420',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:42:23Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={rebuild='server'},migration_context=MigrationContext,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=Flavor(11),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='815b65aa5ad9442db0a087c646f96bbd',ramdisk_id='',reservation_id='r-usz98205',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1898059859',owner_user_name='tempest-ServerActionsTestJSON-1898059859-project-member'},tags=<?>,task_state='resize_reverting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:43:55Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a7f422f1a05a4a92ba387e424850b360',uuid=48977eb2-ce43-4577-a1cb-340e35703dbd,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='resized') vif={"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:44:06 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converting VIF {"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:44:06 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:44:06 user nova-compute[72081]: DEBUG os_vif [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:44:06 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:06 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:44:06 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:44:06 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:06 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap18f9f68c-ea, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:44:06 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap18f9f68c-ea, col_values=(('external_ids', {'iface-id': '18f9f68c-eadd-4582-8f16-b88a13496852', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:b3:c5:ca', 'vm-uuid': '48977eb2-ce43-4577-a1cb-340e35703dbd'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:44:06 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:06 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:44:06 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:06 user nova-compute[72081]: INFO os_vif [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea')
Apr 12 03:44:06 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:06 user nova-compute[72081]: DEBUG nova.compute.manager [req-e83b29e0-201d-4479-8fda-f9f2db80f384 req-f57c2636-f3af-4941-aa72-beac333500e3 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:44:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e83b29e0-201d-4479-8fda-f9f2db80f384 req-f57c2636-f3af-4941-aa72-beac333500e3 service nova] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:44:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e83b29e0-201d-4479-8fda-f9f2db80f384 req-f57c2636-f3af-4941-aa72-beac333500e3 service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:44:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e83b29e0-201d-4479-8fda-f9f2db80f384 req-f57c2636-f3af-4941-aa72-beac333500e3 service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:44:06 user nova-compute[72081]: DEBUG nova.compute.manager [req-e83b29e0-201d-4479-8fda-f9f2db80f384 req-f57c2636-f3af-4941-aa72-beac333500e3 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] No waiting events found dispatching network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:44:06 user nova-compute[72081]: WARNING nova.compute.manager [req-e83b29e0-201d-4479-8fda-f9f2db80f384 req-f57c2636-f3af-4941-aa72-beac333500e3 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received unexpected event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 for instance with vm_state stopped and task_state None.
Apr 12 03:44:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-c032f582-2fdc-4c9e-a502-1a79ede17808 req-98c18168-faa4-4d68-999c-5d9080203395 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:44:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c032f582-2fdc-4c9e-a502-1a79ede17808 req-98c18168-faa4-4d68-999c-5d9080203395 service nova] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:44:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c032f582-2fdc-4c9e-a502-1a79ede17808 req-98c18168-faa4-4d68-999c-5d9080203395 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:44:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c032f582-2fdc-4c9e-a502-1a79ede17808 req-98c18168-faa4-4d68-999c-5d9080203395 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:44:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-c032f582-2fdc-4c9e-a502-1a79ede17808 req-98c18168-faa4-4d68-999c-5d9080203395 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] No waiting events found dispatching network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:44:07 user nova-compute[72081]: WARNING nova.compute.manager [req-c032f582-2fdc-4c9e-a502-1a79ede17808 req-98c18168-faa4-4d68-999c-5d9080203395 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received unexpected event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 for instance with vm_state resized and task_state resize_reverting.
Apr 12 03:44:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Removed pending event for 48977eb2-ce43-4577-a1cb-340e35703dbd due to event {{(pid=72081) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 12 03:44:09 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271049.5764453, 48977eb2-ce43-4577-a1cb-340e35703dbd => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:44:09 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] VM Resumed (Lifecycle Event)
Apr 12 03:44:09 user nova-compute[72081]: DEBUG nova.compute.manager [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:44:09 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Instance running successfully.
Apr 12 03:44:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] finish_revert_migration finished successfully. {{(pid=72081) finish_revert_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11836}}
Apr 12 03:44:09 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:44:09 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: resized, current task_state: resize_reverting, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:44:09 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] During sync_power_state the instance has a pending task (resize_reverting). Skip.
Apr 12 03:44:09 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271049.5818477, 48977eb2-ce43-4577-a1cb-340e35703dbd => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:44:09 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] VM Started (Lifecycle Event)
Apr 12 03:44:09 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:44:09 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:44:09 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:44:09 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Synchronizing instance power state after lifecycle event "Started"; current vm_state: resized, current task_state: resize_reverting, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:44:09 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] During sync_power_state the instance has a pending task (resize_reverting). Skip.
Apr 12 03:44:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-25a66390-af93-4fa5-bf9c-b3a309ccfcf1 req-8facec7e-38c2-4940-aba4-e4a522aa8c91 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:44:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-25a66390-af93-4fa5-bf9c-b3a309ccfcf1 req-8facec7e-38c2-4940-aba4-e4a522aa8c91 service nova] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:44:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-25a66390-af93-4fa5-bf9c-b3a309ccfcf1 req-8facec7e-38c2-4940-aba4-e4a522aa8c91 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:44:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-25a66390-af93-4fa5-bf9c-b3a309ccfcf1 req-8facec7e-38c2-4940-aba4-e4a522aa8c91 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:44:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-25a66390-af93-4fa5-bf9c-b3a309ccfcf1 req-8facec7e-38c2-4940-aba4-e4a522aa8c91 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] No waiting events found dispatching network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:44:09 user nova-compute[72081]: WARNING nova.compute.manager [req-25a66390-af93-4fa5-bf9c-b3a309ccfcf1 req-8facec7e-38c2-4940-aba4-e4a522aa8c91 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received unexpected event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 for instance with vm_state resized and task_state resize_reverting.
Apr 12 03:44:09 user nova-compute[72081]: INFO nova.compute.manager [None req-3dbb368b-ed45-43c2-bc6d-6a7844654de0 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Updating instance to original state: 'active'
Apr 12 03:44:10 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:44:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:11 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:44:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:44:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:44:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:44:11 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:44:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:11 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:12 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdg (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:44:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk --force-share --output=json" returned: 0 in 0.216s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:12 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdf (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-45e5ef51-8e5a-418c-9e1a-3f1a988fe6b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-45e5ef51-8e5a-418c-9e1a-3f1a988fe6b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-45e5ef51-8e5a-418c-9e1a-3f1a988fe6b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-45e5ef51-8e5a-418c-9e1a-3f1a988fe6b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-45e5ef51-8e5a-418c-9e1a-3f1a988fe6b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:44:13 user nova-compute[72081]: INFO nova.compute.manager [None req-45e5ef51-8e5a-418c-9e1a-3f1a988fe6b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Terminating instance
Apr 12 03:44:13 user nova-compute[72081]: DEBUG nova.compute.manager [None req-45e5ef51-8e5a-418c-9e1a-3f1a988fe6b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0/disk --force-share --output=json" returned: 0 in 0.151s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/614495f2-f108-4759-be60-bc6cc689c5b7/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG nova.compute.manager [req-05232a6f-e1e5-40b3-aacf-e6ce7fa549a8 req-a8bdcbaf-778c-4ee0-b097-14eb65b84eee service nova] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Received event network-vif-unplugged-979f4007-c0b3-408e-9f1c-a62c05af6417 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-05232a6f-e1e5-40b3-aacf-e6ce7fa549a8 req-a8bdcbaf-778c-4ee0-b097-14eb65b84eee service nova] Acquiring lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-05232a6f-e1e5-40b3-aacf-e6ce7fa549a8 req-a8bdcbaf-778c-4ee0-b097-14eb65b84eee service nova] Lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-05232a6f-e1e5-40b3-aacf-e6ce7fa549a8 req-a8bdcbaf-778c-4ee0-b097-14eb65b84eee service nova] Lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG nova.compute.manager [req-05232a6f-e1e5-40b3-aacf-e6ce7fa549a8 req-a8bdcbaf-778c-4ee0-b097-14eb65b84eee service nova] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] No waiting events found dispatching network-vif-unplugged-979f4007-c0b3-408e-9f1c-a62c05af6417 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG nova.compute.manager [req-05232a6f-e1e5-40b3-aacf-e6ce7fa549a8 req-a8bdcbaf-778c-4ee0-b097-14eb65b84eee service nova] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Received event network-vif-unplugged-979f4007-c0b3-408e-9f1c-a62c05af6417 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/614495f2-f108-4759-be60-bc6cc689c5b7/disk.rescue --force-share --output=json" returned: 0 in 0.148s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/614495f2-f108-4759-be60-bc6cc689c5b7/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/614495f2-f108-4759-be60-bc6cc689c5b7/disk.rescue --force-share --output=json" returned: 0 in 0.150s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:14 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Instance destroyed successfully.
Apr 12 03:44:14 user nova-compute[72081]: DEBUG nova.objects.instance [None req-45e5ef51-8e5a-418c-9e1a-3f1a988fe6b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lazy-loading 'resources' on Instance uuid f73fa883-75f0-4593-a89c-d009c6d4ebf0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-45e5ef51-8e5a-418c-9e1a-3f1a988fe6b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:38:34Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-1062487719',display_name='tempest-AttachVolumeNegativeTest-server-1062487719',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-1062487719',id=23,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS13JiPBnpwy6hWDJrnaZHwuwdL5vH5nyF6/MTnPZMJQcLak2kXYJ42hH5AmlQraI/A/u9o/w1zAFpUYDoJwJLsZ7venHMIO1i9s9kjpJA5JVZMg0XUxqXqwfxeNoHNZQ==',key_name='tempest-keypair-1411992271',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:38:40Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='d13546149a4e44e1b782b99e9eb452e8',ramdisk_id='',reservation_id='r-5zkadp10',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeNegativeTest-494105310',owner_user_name='tempest-AttachVolumeNegativeTest-494105310-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:38:41Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d52637cca2114e5e99e688ee371c835a',uuid=f73fa883-75f0-4593-a89c-d009c6d4ebf0,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "979f4007-c0b3-408e-9f1c-a62c05af6417", "address": "fa:16:3e:fd:fc:16", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.81", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap979f4007-c0", "ovs_interfaceid": "979f4007-c0b3-408e-9f1c-a62c05af6417", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-45e5ef51-8e5a-418c-9e1a-3f1a988fe6b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Converting VIF {"id": "979f4007-c0b3-408e-9f1c-a62c05af6417", "address": "fa:16:3e:fd:fc:16", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.81", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap979f4007-c0", "ovs_interfaceid": "979f4007-c0b3-408e-9f1c-a62c05af6417", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-45e5ef51-8e5a-418c-9e1a-3f1a988fe6b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:fd:fc:16,bridge_name='br-int',has_traffic_filtering=True,id=979f4007-c0b3-408e-9f1c-a62c05af6417,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap979f4007-c0') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG os_vif [None req-45e5ef51-8e5a-418c-9e1a-3f1a988fe6b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:fd:fc:16,bridge_name='br-int',has_traffic_filtering=True,id=979f4007-c0b3-408e-9f1c-a62c05af6417,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap979f4007-c0') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap979f4007-c0, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sde (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:14 user nova-compute[72081]: INFO os_vif [None req-45e5ef51-8e5a-418c-9e1a-3f1a988fe6b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:fd:fc:16,bridge_name='br-int',has_traffic_filtering=True,id=979f4007-c0b3-408e-9f1c-a62c05af6417,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap979f4007-c0')
Apr 12 03:44:14 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-45e5ef51-8e5a-418c-9e1a-3f1a988fe6b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Deleting instance files /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0_del
Apr 12 03:44:14 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-45e5ef51-8e5a-418c-9e1a-3f1a988fe6b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Deletion of /opt/stack/data/nova/instances/f73fa883-75f0-4593-a89c-d009c6d4ebf0_del complete
Apr 12 03:44:14 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:44:14 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:44:14 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=7861MB free_disk=25.912498474121094GB free_vcpus=4 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:44:14 user nova-compute[72081]: INFO nova.compute.manager [None req-45e5ef51-8e5a-418c-9e1a-3f1a988fe6b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Took 1.32 seconds to destroy the instance on the hypervisor.
Apr 12 03:44:14 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-45e5ef51-8e5a-418c-9e1a-3f1a988fe6b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 2d0904c3-74e0-47bb-a832-758a7035b2f5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance bcd267d2-5b2c-4ed2-b634-0210fae94dfd actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance f73fa883-75f0-4593-a89c-d009c6d4ebf0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 3d4407f4-e078-4777-a110-cb3f922be2e8 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 614495f2-f108-4759-be60-bc6cc689c5b7 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 606425bb-8968-41f9-b3a5-80b6fd188a67 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 48977eb2-ce43-4577-a1cb-340e35703dbd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 128, 'DISK_GB': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 93e45116-bb5c-40ca-9fc5-1c5c40ff687e actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=1664MB phys_disk=40GB used_disk=6GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:44:14 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:44:15 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:44:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.586s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:44:15 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:44:15 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Took 0.93 seconds to deallocate network for instance.
Apr 12 03:44:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-45e5ef51-8e5a-418c-9e1a-3f1a988fe6b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:44:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-45e5ef51-8e5a-418c-9e1a-3f1a988fe6b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:44:15 user nova-compute[72081]: DEBUG nova.compute.manager [req-417f1cb7-6cbd-4e4e-8388-2e482f31d6b4 req-9ba0fb8a-0d39-4479-8c50-0eb361c2487c service nova] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Received event network-vif-plugged-979f4007-c0b3-408e-9f1c-a62c05af6417 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:44:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-417f1cb7-6cbd-4e4e-8388-2e482f31d6b4 req-9ba0fb8a-0d39-4479-8c50-0eb361c2487c service nova] Acquiring lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:44:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-417f1cb7-6cbd-4e4e-8388-2e482f31d6b4 req-9ba0fb8a-0d39-4479-8c50-0eb361c2487c service nova] Lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:44:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-417f1cb7-6cbd-4e4e-8388-2e482f31d6b4 req-9ba0fb8a-0d39-4479-8c50-0eb361c2487c service nova] Lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:44:15 user nova-compute[72081]: DEBUG nova.compute.manager [req-417f1cb7-6cbd-4e4e-8388-2e482f31d6b4 req-9ba0fb8a-0d39-4479-8c50-0eb361c2487c service nova] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] No waiting events found dispatching network-vif-plugged-979f4007-c0b3-408e-9f1c-a62c05af6417 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:44:15 user nova-compute[72081]: WARNING nova.compute.manager [req-417f1cb7-6cbd-4e4e-8388-2e482f31d6b4 req-9ba0fb8a-0d39-4479-8c50-0eb361c2487c service nova] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Received unexpected event network-vif-plugged-979f4007-c0b3-408e-9f1c-a62c05af6417 for instance with vm_state deleted and task_state None.
Apr 12 03:44:15 user nova-compute[72081]: DEBUG nova.compute.manager [req-417f1cb7-6cbd-4e4e-8388-2e482f31d6b4 req-9ba0fb8a-0d39-4479-8c50-0eb361c2487c service nova] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Received event network-vif-deleted-979f4007-c0b3-408e-9f1c-a62c05af6417 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:44:15 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-45e5ef51-8e5a-418c-9e1a-3f1a988fe6b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:44:15 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-45e5ef51-8e5a-418c-9e1a-3f1a988fe6b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:44:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-45e5ef51-8e5a-418c-9e1a-3f1a988fe6b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.352s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:44:15 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-45e5ef51-8e5a-418c-9e1a-3f1a988fe6b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Deleted allocations for instance f73fa883-75f0-4593-a89c-d009c6d4ebf0
Apr 12 03:44:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-45e5ef51-8e5a-418c-9e1a-3f1a988fe6b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "f73fa883-75f0-4593-a89c-d009c6d4ebf0" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.833s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:44:16 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:44:16 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:44:16 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:44:16 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:44:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-3d4407f4-e078-4777-a110-cb3f922be2e8" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:44:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-3d4407f4-e078-4777-a110-cb3f922be2e8" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:44:16 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 03:44:16 user nova-compute[72081]: DEBUG nova.compute.manager [req-61a6588a-7ffe-4d74-a7ac-44ab98b95f95 req-3d094e83-8da3-43a2-88fe-291adb4a63c7 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Received event network-changed-c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:44:16 user nova-compute[72081]: DEBUG nova.compute.manager [req-61a6588a-7ffe-4d74-a7ac-44ab98b95f95 req-3d094e83-8da3-43a2-88fe-291adb4a63c7 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Refreshing instance network info cache due to event network-changed-c904583a-ea96-45c9-ad62-d5e0cc68031f. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:44:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-61a6588a-7ffe-4d74-a7ac-44ab98b95f95 req-3d094e83-8da3-43a2-88fe-291adb4a63c7 service nova] Acquiring lock "refresh_cache-2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:44:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-61a6588a-7ffe-4d74-a7ac-44ab98b95f95 req-3d094e83-8da3-43a2-88fe-291adb4a63c7 service nova] Acquired lock "refresh_cache-2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:44:16 user nova-compute[72081]: DEBUG nova.network.neutron [req-61a6588a-7ffe-4d74-a7ac-44ab98b95f95 req-3d094e83-8da3-43a2-88fe-291adb4a63c7 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Refreshing network info cache for port c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:44:17 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Updating instance_info_cache with network_info: [{"id": "aec6d617-eb2b-42b3-8cce-f21de7c1b91c", "address": "fa:16:3e:92:a6:05", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapaec6d617-eb", "ovs_interfaceid": "aec6d617-eb2b-42b3-8cce-f21de7c1b91c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:44:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-3d4407f4-e078-4777-a110-cb3f922be2e8" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:44:17 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 03:44:17 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:44:17 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:44:17 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:44:17 user nova-compute[72081]: DEBUG nova.network.neutron [req-61a6588a-7ffe-4d74-a7ac-44ab98b95f95 req-3d094e83-8da3-43a2-88fe-291adb4a63c7 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Updated VIF entry in instance network info cache for port c904583a-ea96-45c9-ad62-d5e0cc68031f. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:44:17 user nova-compute[72081]: DEBUG nova.network.neutron [req-61a6588a-7ffe-4d74-a7ac-44ab98b95f95 req-3d094e83-8da3-43a2-88fe-291adb4a63c7 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Updating instance_info_cache with network_info: [{"id": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "address": "fa:16:3e:84:c4:9d", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.10", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc904583a-ea", "ovs_interfaceid": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:44:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:44:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:44:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-61a6588a-7ffe-4d74-a7ac-44ab98b95f95 req-3d094e83-8da3-43a2-88fe-291adb4a63c7 service nova] Releasing lock "refresh_cache-2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:44:17 user nova-compute[72081]: INFO nova.compute.manager [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Detaching volume 8568ebe9-126e-4516-87a9-cd53c4b520c3
Apr 12 03:44:17 user nova-compute[72081]: INFO nova.virt.block_device [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Attempting to driver detach volume 8568ebe9-126e-4516-87a9-cd53c4b520c3 from mountpoint /dev/vdb
Apr 12 03:44:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Attempting to detach device vdb from instance 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170 from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 03:44:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:44:17 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:44:17 user nova-compute[72081]:   <source dev="/dev/sdf"/>
Apr 12 03:44:17 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:44:17 user nova-compute[72081]:   <serial>8568ebe9-126e-4516-87a9-cd53c4b520c3</serial>
Apr 12 03:44:17 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 03:44:17 user nova-compute[72081]: </disk>
Apr 12 03:44:17 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:44:17 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Successfully detached device vdb from instance 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170 from the persistent domain config.
Apr 12 03:44:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170 from the live domain config. {{(pid=72081) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 12 03:44:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:44:17 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:44:17 user nova-compute[72081]:   <source dev="/dev/sdf"/>
Apr 12 03:44:17 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:44:17 user nova-compute[72081]:   <serial>8568ebe9-126e-4516-87a9-cd53c4b520c3</serial>
Apr 12 03:44:17 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 03:44:17 user nova-compute[72081]: </disk>
Apr 12 03:44:17 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Received event <DeviceRemovedEvent: 1681271058.0267158, 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72081) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170 {{(pid=72081) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 12 03:44:18 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Successfully detached device vdb from instance 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170 from the live domain config.
Apr 12 03:44:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'nDKzpQvs', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:44:18 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:44:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[914632cb-b5b5-49bf-a1a2-284fc4b16c3e]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:44:18 user nova-compute[72081]: No targets found.
Apr 12 03:44:18 user nova-compute[72081]: iSNS:
Apr 12 03:44:18 user nova-compute[72081]: No targets found.
Apr 12 03:44:18 user nova-compute[72081]: STATIC:
Apr 12 03:44:18 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27
Apr 12 03:44:18 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:44:18 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:44:18 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:44:18 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:44:18 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:44:18 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e
Apr 12 03:44:18 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:44:18 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:44:18 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068
Apr 12 03:44:18 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:44:18 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:44:18 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3
Apr 12 03:44:18 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:44:18 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:44:18 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742
Apr 12 03:44:18 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:44:18 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:44:18 user nova-compute[72081]: FIRMWARE:
Apr 12 03:44:18 user nova-compute[72081]: No targets found.
Apr 12 03:44:18 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:44:18 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:44:18 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[35212063-401b-487c-9517-18f50ab1a500]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.042s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6aeaefaf-11b9-49f3-8b36-e0fb32040bc9]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:44:18 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:44:18 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:44:18 user nova-compute[72081]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:44:18 user nova-compute[72081]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:44:18 user nova-compute[72081]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:44:18 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:44:18 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:44:18 user nova-compute[72081]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 (non-flash)
Apr 12 03:44:18 user nova-compute[72081]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:44:18 user nova-compute[72081]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:44:18 user nova-compute[72081]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:44:18 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fba3b6dd0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3'): ({'sdf'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Removing single pathed devices sdf {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[34aea086-e1c0-40c2-a55f-1807766ebf64]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdf {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdf" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ee3a8af9-ddbc-4d22-b450-76111dd658fe]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Flushing IO for device /dev/sdf {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdf {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdf" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[86720bb6-356b-4d38-ae73-3f71083a89e3]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Remove SCSI device /dev/sdf with /sys/block/sdf/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdf/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdf/device/delete" returned: 0 in 0.044s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1415ba6f-7788-4f51-80b8-5189a8a3158a]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Checking to see if SCSI volumes sdf have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] SCSI volumes sdf have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b59b68d-3058-46d9-bf08-62b7e5d03d7d]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 -p 172.16.0.220:3260 --logout" returned: 0 in 0.064s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c7b6fcf6-db9d-4d8a-b95d-8a092fd6e6e3]: (4, ('Logging out of session [sid: 30, target: iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3, portal: 172.16.0.220,3260]\nLogout of [sid: 30, target: iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 30, target: iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3, portal: 172.16.0.220,3260]
Apr 12 03:44:18 user nova-compute[72081]: Logout of [sid: 30, target: iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3, portal: 172.16.0.220,3260] successful.
Apr 12 03:44:18 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8568ebe9-126e-4516-87a9-cd53c4b520c3 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.029s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e63c4432-fb69-46cf-b063-49bbd4ff5036]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.346s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] <== disconnect_volume: return (347ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG nova.compute.manager [req-3998ba97-e7e1-4d71-92f4-7254c3476003 req-dce0a49d-94e1-4864-9983-b55e3de2ac52 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Received event network-changed-c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG nova.compute.manager [req-3998ba97-e7e1-4d71-92f4-7254c3476003 req-dce0a49d-94e1-4864-9983-b55e3de2ac52 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Refreshing instance network info cache due to event network-changed-c904583a-ea96-45c9-ad62-d5e0cc68031f. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3998ba97-e7e1-4d71-92f4-7254c3476003 req-dce0a49d-94e1-4864-9983-b55e3de2ac52 service nova] Acquiring lock "refresh_cache-2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3998ba97-e7e1-4d71-92f4-7254c3476003 req-dce0a49d-94e1-4864-9983-b55e3de2ac52 service nova] Acquired lock "refresh_cache-2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:44:18 user nova-compute[72081]: DEBUG nova.network.neutron [req-3998ba97-e7e1-4d71-92f4-7254c3476003 req-dce0a49d-94e1-4864-9983-b55e3de2ac52 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Refreshing network info cache for port c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:44:19 user nova-compute[72081]: DEBUG nova.network.neutron [req-3998ba97-e7e1-4d71-92f4-7254c3476003 req-dce0a49d-94e1-4864-9983-b55e3de2ac52 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Updated VIF entry in instance network info cache for port c904583a-ea96-45c9-ad62-d5e0cc68031f. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:44:19 user nova-compute[72081]: DEBUG nova.network.neutron [req-3998ba97-e7e1-4d71-92f4-7254c3476003 req-dce0a49d-94e1-4864-9983-b55e3de2ac52 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Updating instance_info_cache with network_info: [{"id": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "address": "fa:16:3e:84:c4:9d", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc904583a-ea", "ovs_interfaceid": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:44:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3998ba97-e7e1-4d71-92f4-7254c3476003 req-dce0a49d-94e1-4864-9983-b55e3de2ac52 service nova] Releasing lock "refresh_cache-2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:44:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:19 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681271044.372399, 606425bb-8968-41f9-b3a5-80b6fd188a67 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:44:19 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] VM Stopped (Lifecycle Event)
Apr 12 03:44:19 user nova-compute[72081]: DEBUG nova.compute.manager [None req-97cedec4-2c3d-4455-9898-57a3e73e1bef None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:44:19 user nova-compute[72081]: DEBUG nova.compute.manager [None req-97cedec4-2c3d-4455-9898-57a3e73e1bef None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: stopped, current task_state: None, current DB power_state: 4, VM power_state: 4 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:44:20 user nova-compute[72081]: DEBUG nova.objects.instance [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lazy-loading 'flavor' on Instance uuid 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:44:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f610b4fd-3b18-4234-bf94-98c54012fcf4 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 2.919s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:44:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:28 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681271053.990685, f73fa883-75f0-4593-a89c-d009c6d4ebf0 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:44:28 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] VM Stopped (Lifecycle Event)
Apr 12 03:44:29 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1d636485-50eb-4764-ad78-96f808589fd2 None None] [instance: f73fa883-75f0-4593-a89c-d009c6d4ebf0] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:44:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:44:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:44:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:45:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:06 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:45:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "a264f3f7-b0ea-482a-a438-909f06a80726" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "a264f3f7-b0ea-482a-a438-909f06a80726" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:45:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:07 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:45:07 user nova-compute[72081]: INFO nova.compute.claims [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Claim successful on node user
Apr 12 03:45:07 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:45:07 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:45:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.383s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:45:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:45:07 user nova-compute[72081]: DEBUG nova.network.neutron [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:45:07 user nova-compute[72081]: DEBUG nova.policy [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd52637cca2114e5e99e688ee371c835a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd13546149a4e44e1b782b99e9eb452e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:45:07 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:45:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG nova.compute.manager [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:45:08 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Creating image(s)
Apr 12 03:45:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "/opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "/opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "/opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.192s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.153s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726/disk 1073741824" returned: 0 in 0.058s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.219s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.177s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Checking if we can resize image /opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG nova.network.neutron [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Successfully created port: e294fdef-5e1a-47fd-bcb1-08b22b4941b0 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Cannot resize image /opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG nova.objects.instance [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lazy-loading 'migration_context' on Instance uuid a264f3f7-b0ea-482a-a438-909f06a80726 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Ensure instance console log exists: /opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.network.neutron [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Successfully updated port: e294fdef-5e1a-47fd-bcb1-08b22b4941b0 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "refresh_cache-a264f3f7-b0ea-482a-a438-909f06a80726" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquired lock "refresh_cache-a264f3f7-b0ea-482a-a438-909f06a80726" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.network.neutron [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-015c4a09-0690-4fe6-86e8-631b18a8fb98 req-d8a8d894-6b09-48b0-9310-3809ec71cc22 service nova] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Received event network-changed-e294fdef-5e1a-47fd-bcb1-08b22b4941b0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-015c4a09-0690-4fe6-86e8-631b18a8fb98 req-d8a8d894-6b09-48b0-9310-3809ec71cc22 service nova] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Refreshing instance network info cache due to event network-changed-e294fdef-5e1a-47fd-bcb1-08b22b4941b0. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-015c4a09-0690-4fe6-86e8-631b18a8fb98 req-d8a8d894-6b09-48b0-9310-3809ec71cc22 service nova] Acquiring lock "refresh_cache-a264f3f7-b0ea-482a-a438-909f06a80726" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.network.neutron [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.network.neutron [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Updating instance_info_cache with network_info: [{"id": "e294fdef-5e1a-47fd-bcb1-08b22b4941b0", "address": "fa:16:3e:4a:45:5f", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape294fdef-5e", "ovs_interfaceid": "e294fdef-5e1a-47fd-bcb1-08b22b4941b0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Releasing lock "refresh_cache-a264f3f7-b0ea-482a-a438-909f06a80726" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.compute.manager [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Instance network_info: |[{"id": "e294fdef-5e1a-47fd-bcb1-08b22b4941b0", "address": "fa:16:3e:4a:45:5f", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape294fdef-5e", "ovs_interfaceid": "e294fdef-5e1a-47fd-bcb1-08b22b4941b0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-015c4a09-0690-4fe6-86e8-631b18a8fb98 req-d8a8d894-6b09-48b0-9310-3809ec71cc22 service nova] Acquired lock "refresh_cache-a264f3f7-b0ea-482a-a438-909f06a80726" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.network.neutron [req-015c4a09-0690-4fe6-86e8-631b18a8fb98 req-d8a8d894-6b09-48b0-9310-3809ec71cc22 service nova] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Refreshing network info cache for port e294fdef-5e1a-47fd-bcb1-08b22b4941b0 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Start _get_guest_xml network_info=[{"id": "e294fdef-5e1a-47fd-bcb1-08b22b4941b0", "address": "fa:16:3e:4a:45:5f", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape294fdef-5e", "ovs_interfaceid": "e294fdef-5e1a-47fd-bcb1-08b22b4941b0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:45:09 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:45:09 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:45:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-1427168436',display_name='tempest-AttachVolumeNegativeTest-server-1427168436',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-1427168436',id=33,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEWW3PcC6A3oaeI+T5mnA66uGmCFTzs3IbsSTkdd3DCArM0rR6yU/WOLMnATsGubN5kt/JcbJyDd6J/OXjO1xRs8Gm7pg5070UpCpDFnH5m0xDJi0dRMwI1KMt4s8ojMvQ==',key_name='tempest-keypair-863037271',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='d13546149a4e44e1b782b99e9eb452e8',ramdisk_id='',reservation_id='r-3fhwenfs',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-494105310',owner_user_name='tempest-AttachVolumeNegativeTest-494105310-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:45:08Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d52637cca2114e5e99e688ee371c835a',uuid=a264f3f7-b0ea-482a-a438-909f06a80726,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e294fdef-5e1a-47fd-bcb1-08b22b4941b0", "address": "fa:16:3e:4a:45:5f", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape294fdef-5e", "ovs_interfaceid": "e294fdef-5e1a-47fd-bcb1-08b22b4941b0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Converting VIF {"id": "e294fdef-5e1a-47fd-bcb1-08b22b4941b0", "address": "fa:16:3e:4a:45:5f", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape294fdef-5e", "ovs_interfaceid": "e294fdef-5e1a-47fd-bcb1-08b22b4941b0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:4a:45:5f,bridge_name='br-int',has_traffic_filtering=True,id=e294fdef-5e1a-47fd-bcb1-08b22b4941b0,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape294fdef-5e') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.objects.instance [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lazy-loading 'pci_devices' on Instance uuid a264f3f7-b0ea-482a-a438-909f06a80726 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:45:09 user nova-compute[72081]:   <uuid>a264f3f7-b0ea-482a-a438-909f06a80726</uuid>
Apr 12 03:45:09 user nova-compute[72081]:   <name>instance-00000021</name>
Apr 12 03:45:09 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:45:09 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:45:09 user nova-compute[72081]:   <metadata>
Apr 12 03:45:09 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:45:09 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:45:09 user nova-compute[72081]:       <nova:name>tempest-AttachVolumeNegativeTest-server-1427168436</nova:name>
Apr 12 03:45:09 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:45:09</nova:creationTime>
Apr 12 03:45:09 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:45:09 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:45:09 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:45:09 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:45:09 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:45:09 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:45:09 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:45:09 user nova-compute[72081]:       <nova:owner>
Apr 12 03:45:09 user nova-compute[72081]:         <nova:user uuid="d52637cca2114e5e99e688ee371c835a">tempest-AttachVolumeNegativeTest-494105310-project-member</nova:user>
Apr 12 03:45:09 user nova-compute[72081]:         <nova:project uuid="d13546149a4e44e1b782b99e9eb452e8">tempest-AttachVolumeNegativeTest-494105310</nova:project>
Apr 12 03:45:09 user nova-compute[72081]:       </nova:owner>
Apr 12 03:45:09 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:45:09 user nova-compute[72081]:       <nova:ports>
Apr 12 03:45:09 user nova-compute[72081]:         <nova:port uuid="e294fdef-5e1a-47fd-bcb1-08b22b4941b0">
Apr 12 03:45:09 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Apr 12 03:45:09 user nova-compute[72081]:         </nova:port>
Apr 12 03:45:09 user nova-compute[72081]:       </nova:ports>
Apr 12 03:45:09 user nova-compute[72081]:     </nova:instance>
Apr 12 03:45:09 user nova-compute[72081]:   </metadata>
Apr 12 03:45:09 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:45:09 user nova-compute[72081]:     <system>
Apr 12 03:45:09 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:45:09 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:45:09 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:45:09 user nova-compute[72081]:       <entry name="serial">a264f3f7-b0ea-482a-a438-909f06a80726</entry>
Apr 12 03:45:09 user nova-compute[72081]:       <entry name="uuid">a264f3f7-b0ea-482a-a438-909f06a80726</entry>
Apr 12 03:45:09 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:45:09 user nova-compute[72081]:     </system>
Apr 12 03:45:09 user nova-compute[72081]:   </sysinfo>
Apr 12 03:45:09 user nova-compute[72081]:   <os>
Apr 12 03:45:09 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:45:09 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:45:09 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:45:09 user nova-compute[72081]:   </os>
Apr 12 03:45:09 user nova-compute[72081]:   <features>
Apr 12 03:45:09 user nova-compute[72081]:     <acpi/>
Apr 12 03:45:09 user nova-compute[72081]:     <apic/>
Apr 12 03:45:09 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:45:09 user nova-compute[72081]:   </features>
Apr 12 03:45:09 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:45:09 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:45:09 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:45:09 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:45:09 user nova-compute[72081]:   </clock>
Apr 12 03:45:09 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:45:09 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:45:09 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:45:09 user nova-compute[72081]:   </cpu>
Apr 12 03:45:09 user nova-compute[72081]:   <devices>
Apr 12 03:45:09 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:45:09 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:45:09 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726/disk"/>
Apr 12 03:45:09 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:45:09 user nova-compute[72081]:     </disk>
Apr 12 03:45:09 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:45:09 user nova-compute[72081]:       <mac address="fa:16:3e:4a:45:5f"/>
Apr 12 03:45:09 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:45:09 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:45:09 user nova-compute[72081]:       <target dev="tape294fdef-5e"/>
Apr 12 03:45:09 user nova-compute[72081]:     </interface>
Apr 12 03:45:09 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:45:09 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726/console.log" append="off"/>
Apr 12 03:45:09 user nova-compute[72081]:     </serial>
Apr 12 03:45:09 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:45:09 user nova-compute[72081]:     <video>
Apr 12 03:45:09 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:45:09 user nova-compute[72081]:     </video>
Apr 12 03:45:09 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:45:09 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:45:09 user nova-compute[72081]:     </rng>
Apr 12 03:45:09 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:45:09 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:45:09 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:45:09 user nova-compute[72081]:     </memballoon>
Apr 12 03:45:09 user nova-compute[72081]:   </devices>
Apr 12 03:45:09 user nova-compute[72081]: </domain>
Apr 12 03:45:09 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:45:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-1427168436',display_name='tempest-AttachVolumeNegativeTest-server-1427168436',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-1427168436',id=33,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEWW3PcC6A3oaeI+T5mnA66uGmCFTzs3IbsSTkdd3DCArM0rR6yU/WOLMnATsGubN5kt/JcbJyDd6J/OXjO1xRs8Gm7pg5070UpCpDFnH5m0xDJi0dRMwI1KMt4s8ojMvQ==',key_name='tempest-keypair-863037271',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='d13546149a4e44e1b782b99e9eb452e8',ramdisk_id='',reservation_id='r-3fhwenfs',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-494105310',owner_user_name='tempest-AttachVolumeNegativeTest-494105310-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:45:08Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d52637cca2114e5e99e688ee371c835a',uuid=a264f3f7-b0ea-482a-a438-909f06a80726,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e294fdef-5e1a-47fd-bcb1-08b22b4941b0", "address": "fa:16:3e:4a:45:5f", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape294fdef-5e", "ovs_interfaceid": "e294fdef-5e1a-47fd-bcb1-08b22b4941b0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Converting VIF {"id": "e294fdef-5e1a-47fd-bcb1-08b22b4941b0", "address": "fa:16:3e:4a:45:5f", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape294fdef-5e", "ovs_interfaceid": "e294fdef-5e1a-47fd-bcb1-08b22b4941b0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:4a:45:5f,bridge_name='br-int',has_traffic_filtering=True,id=e294fdef-5e1a-47fd-bcb1-08b22b4941b0,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape294fdef-5e') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG os_vif [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:4a:45:5f,bridge_name='br-int',has_traffic_filtering=True,id=e294fdef-5e1a-47fd-bcb1-08b22b4941b0,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape294fdef-5e') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tape294fdef-5e, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tape294fdef-5e, col_values=(('external_ids', {'iface-id': 'e294fdef-5e1a-47fd-bcb1-08b22b4941b0', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:4a:45:5f', 'vm-uuid': 'a264f3f7-b0ea-482a-a438-909f06a80726'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:45:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:09 user nova-compute[72081]: INFO os_vif [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:4a:45:5f,bridge_name='br-int',has_traffic_filtering=True,id=e294fdef-5e1a-47fd-bcb1-08b22b4941b0,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape294fdef-5e')
Apr 12 03:45:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:45:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] No VIF found with MAC fa:16:3e:4a:45:5f, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:45:10 user nova-compute[72081]: DEBUG nova.network.neutron [req-015c4a09-0690-4fe6-86e8-631b18a8fb98 req-d8a8d894-6b09-48b0-9310-3809ec71cc22 service nova] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Updated VIF entry in instance network info cache for port e294fdef-5e1a-47fd-bcb1-08b22b4941b0. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:45:10 user nova-compute[72081]: DEBUG nova.network.neutron [req-015c4a09-0690-4fe6-86e8-631b18a8fb98 req-d8a8d894-6b09-48b0-9310-3809ec71cc22 service nova] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Updating instance_info_cache with network_info: [{"id": "e294fdef-5e1a-47fd-bcb1-08b22b4941b0", "address": "fa:16:3e:4a:45:5f", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape294fdef-5e", "ovs_interfaceid": "e294fdef-5e1a-47fd-bcb1-08b22b4941b0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:45:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-015c4a09-0690-4fe6-86e8-631b18a8fb98 req-d8a8d894-6b09-48b0-9310-3809ec71cc22 service nova] Releasing lock "refresh_cache-a264f3f7-b0ea-482a-a438-909f06a80726" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:45:10 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:45:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:45:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b7bbd800-b60a-40c0-ac20-bfbb50b37591 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b7bbd800-b60a-40c0-ac20-bfbb50b37591 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b7bbd800-b60a-40c0-ac20-bfbb50b37591 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b7bbd800-b60a-40c0-ac20-bfbb50b37591 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b7bbd800-b60a-40c0-ac20-bfbb50b37591 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:11 user nova-compute[72081]: INFO nova.compute.manager [None req-b7bbd800-b60a-40c0-ac20-bfbb50b37591 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Terminating instance
Apr 12 03:45:11 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b7bbd800-b60a-40c0-ac20-bfbb50b37591 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:45:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:11 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:45:11 user nova-compute[72081]: DEBUG nova.compute.manager [req-1c31f452-aad6-4fd3-a74d-5b7f8fa327a7 req-fd1681b5-2bfd-442a-a2dc-fb20cb12fb78 service nova] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Received event network-vif-plugged-e294fdef-5e1a-47fd-bcb1-08b22b4941b0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:45:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1c31f452-aad6-4fd3-a74d-5b7f8fa327a7 req-fd1681b5-2bfd-442a-a2dc-fb20cb12fb78 service nova] Acquiring lock "a264f3f7-b0ea-482a-a438-909f06a80726-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1c31f452-aad6-4fd3-a74d-5b7f8fa327a7 req-fd1681b5-2bfd-442a-a2dc-fb20cb12fb78 service nova] Lock "a264f3f7-b0ea-482a-a438-909f06a80726-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1c31f452-aad6-4fd3-a74d-5b7f8fa327a7 req-fd1681b5-2bfd-442a-a2dc-fb20cb12fb78 service nova] Lock "a264f3f7-b0ea-482a-a438-909f06a80726-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:11 user nova-compute[72081]: DEBUG nova.compute.manager [req-1c31f452-aad6-4fd3-a74d-5b7f8fa327a7 req-fd1681b5-2bfd-442a-a2dc-fb20cb12fb78 service nova] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] No waiting events found dispatching network-vif-plugged-e294fdef-5e1a-47fd-bcb1-08b22b4941b0 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:45:11 user nova-compute[72081]: WARNING nova.compute.manager [req-1c31f452-aad6-4fd3-a74d-5b7f8fa327a7 req-fd1681b5-2bfd-442a-a2dc-fb20cb12fb78 service nova] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Received unexpected event network-vif-plugged-e294fdef-5e1a-47fd-bcb1-08b22b4941b0 for instance with vm_state building and task_state spawning.
Apr 12 03:45:12 user nova-compute[72081]: DEBUG nova.compute.manager [req-b982062c-d07f-4195-a099-cd5faac79be4 req-03cb70dd-ef0c-4f4a-a396-dee37205affc service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Received event network-vif-unplugged-c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:45:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b982062c-d07f-4195-a099-cd5faac79be4 req-03cb70dd-ef0c-4f4a-a396-dee37205affc service nova] Acquiring lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b982062c-d07f-4195-a099-cd5faac79be4 req-03cb70dd-ef0c-4f4a-a396-dee37205affc service nova] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b982062c-d07f-4195-a099-cd5faac79be4 req-03cb70dd-ef0c-4f4a-a396-dee37205affc service nova] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:12 user nova-compute[72081]: DEBUG nova.compute.manager [req-b982062c-d07f-4195-a099-cd5faac79be4 req-03cb70dd-ef0c-4f4a-a396-dee37205affc service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] No waiting events found dispatching network-vif-unplugged-c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:45:12 user nova-compute[72081]: DEBUG nova.compute.manager [req-b982062c-d07f-4195-a099-cd5faac79be4 req-03cb70dd-ef0c-4f4a-a396-dee37205affc service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Received event network-vif-unplugged-c904583a-ea96-45c9-ad62-d5e0cc68031f for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:45:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:12 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Instance destroyed successfully.
Apr 12 03:45:12 user nova-compute[72081]: DEBUG nova.objects.instance [None req-b7bbd800-b60a-40c0-ac20-bfbb50b37591 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lazy-loading 'resources' on Instance uuid 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:45:12 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-b7bbd800-b60a-40c0-ac20-bfbb50b37591 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:40:17Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-470315255',display_name='tempest-TestMinimumBasicScenario-server-470315255',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-470315255',id=27,image_ref='e198a023-7244-4f4f-b524-67b3b8abbd94',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHPcbdFPNwSacXUiuG3oK16SVNbJsO/MbG4r56gwQ5WdhiWutK0MgzDXL9ED179Ry++/GuDPfwLne43xpH8kd84JNZdPdBD50aHcS2/YXE5ZEXdAfa8QEldhEXCO3fgFsQ==',key_name='tempest-TestMinimumBasicScenario-1199406481',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:40:25Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='05e1e25933cd42df98fb060b8d2701dc',ramdisk_id='',reservation_id='r-220aqgyr',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e198a023-7244-4f4f-b524-67b3b8abbd94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-TestMinimumBasicScenario-1605347971',owner_user_name='tempest-TestMinimumBasicScenario-1605347971-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:42:34Z,user_data=None,user_id='3f8024260bf14d9995e67cb0d1bc8ddc',uuid=2d7f71ac-a671-4feb-8c2d-4cbcb84f7170,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "address": "fa:16:3e:84:c4:9d", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc904583a-ea", "ovs_interfaceid": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:45:12 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b7bbd800-b60a-40c0-ac20-bfbb50b37591 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Converting VIF {"id": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "address": "fa:16:3e:84:c4:9d", "network": {"id": "91a7501a-44a5-4d48-a1f7-e1195ac99607", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-50878364-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "05e1e25933cd42df98fb060b8d2701dc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc904583a-ea", "ovs_interfaceid": "c904583a-ea96-45c9-ad62-d5e0cc68031f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:45:12 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b7bbd800-b60a-40c0-ac20-bfbb50b37591 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:84:c4:9d,bridge_name='br-int',has_traffic_filtering=True,id=c904583a-ea96-45c9-ad62-d5e0cc68031f,network=Network(91a7501a-44a5-4d48-a1f7-e1195ac99607),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc904583a-ea') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:45:12 user nova-compute[72081]: DEBUG os_vif [None req-b7bbd800-b60a-40c0-ac20-bfbb50b37591 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:84:c4:9d,bridge_name='br-int',has_traffic_filtering=True,id=c904583a-ea96-45c9-ad62-d5e0cc68031f,network=Network(91a7501a-44a5-4d48-a1f7-e1195ac99607),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc904583a-ea') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:45:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapc904583a-ea, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:45:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:45:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:12 user nova-compute[72081]: INFO os_vif [None req-b7bbd800-b60a-40c0-ac20-bfbb50b37591 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:84:c4:9d,bridge_name='br-int',has_traffic_filtering=True,id=c904583a-ea96-45c9-ad62-d5e0cc68031f,network=Network(91a7501a-44a5-4d48-a1f7-e1195ac99607),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc904583a-ea')
Apr 12 03:45:12 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-b7bbd800-b60a-40c0-ac20-bfbb50b37591 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Deleting instance files /opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170_del
Apr 12 03:45:12 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-b7bbd800-b60a-40c0-ac20-bfbb50b37591 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Deletion of /opt/stack/data/nova/instances/2d7f71ac-a671-4feb-8c2d-4cbcb84f7170_del complete
Apr 12 03:45:12 user nova-compute[72081]: INFO nova.compute.manager [None req-b7bbd800-b60a-40c0-ac20-bfbb50b37591 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Took 1.11 seconds to destroy the instance on the hypervisor.
Apr 12 03:45:12 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-b7bbd800-b60a-40c0-ac20-bfbb50b37591 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:45:12 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:45:12 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:45:13 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Took 0.51 seconds to deallocate network for instance.
Apr 12 03:45:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b7bbd800-b60a-40c0-ac20-bfbb50b37591 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b7bbd800-b60a-40c0-ac20-bfbb50b37591 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-b7bbd800-b60a-40c0-ac20-bfbb50b37591 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-b7bbd800-b60a-40c0-ac20-bfbb50b37591 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b7bbd800-b60a-40c0-ac20-bfbb50b37591 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.308s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:13 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-b7bbd800-b60a-40c0-ac20-bfbb50b37591 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Deleted allocations for instance 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170
Apr 12 03:45:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b7bbd800-b60a-40c0-ac20-bfbb50b37591 tempest-TestMinimumBasicScenario-1605347971 tempest-TestMinimumBasicScenario-1605347971-project-member] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.161s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271113.6548674, a264f3f7-b0ea-482a-a438-909f06a80726 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:45:13 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] VM Resumed (Lifecycle Event)
Apr 12 03:45:13 user nova-compute[72081]: DEBUG nova.compute.manager [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:45:13 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Instance spawned successfully.
Apr 12 03:45:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG nova.compute.manager [req-01452ac2-38fb-407b-8b1a-ad13541c325c req-36bc0dd0-d5b0-4872-ada3-e64f7d7b0058 service nova] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Received event network-vif-plugged-e294fdef-5e1a-47fd-bcb1-08b22b4941b0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-01452ac2-38fb-407b-8b1a-ad13541c325c req-36bc0dd0-d5b0-4872-ada3-e64f7d7b0058 service nova] Acquiring lock "a264f3f7-b0ea-482a-a438-909f06a80726-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-01452ac2-38fb-407b-8b1a-ad13541c325c req-36bc0dd0-d5b0-4872-ada3-e64f7d7b0058 service nova] Lock "a264f3f7-b0ea-482a-a438-909f06a80726-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-01452ac2-38fb-407b-8b1a-ad13541c325c req-36bc0dd0-d5b0-4872-ada3-e64f7d7b0058 service nova] Lock "a264f3f7-b0ea-482a-a438-909f06a80726-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG nova.compute.manager [req-01452ac2-38fb-407b-8b1a-ad13541c325c req-36bc0dd0-d5b0-4872-ada3-e64f7d7b0058 service nova] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] No waiting events found dispatching network-vif-plugged-e294fdef-5e1a-47fd-bcb1-08b22b4941b0 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:45:13 user nova-compute[72081]: WARNING nova.compute.manager [req-01452ac2-38fb-407b-8b1a-ad13541c325c req-36bc0dd0-d5b0-4872-ada3-e64f7d7b0058 service nova] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Received unexpected event network-vif-plugged-e294fdef-5e1a-47fd-bcb1-08b22b4941b0 for instance with vm_state building and task_state spawning.
Apr 12 03:45:13 user nova-compute[72081]: DEBUG nova.compute.manager [req-01452ac2-38fb-407b-8b1a-ad13541c325c req-36bc0dd0-d5b0-4872-ada3-e64f7d7b0058 service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Received event network-vif-deleted-c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:45:13 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:45:13 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271113.658422, a264f3f7-b0ea-482a-a438-909f06a80726 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:45:13 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] VM Started (Lifecycle Event)
Apr 12 03:45:13 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:45:13 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:45:13 user nova-compute[72081]: INFO nova.compute.manager [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Took 5.76 seconds to spawn the instance on the hypervisor.
Apr 12 03:45:13 user nova-compute[72081]: DEBUG nova.compute.manager [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:45:13 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:45:13 user nova-compute[72081]: INFO nova.compute.manager [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Took 6.80 seconds to build instance.
Apr 12 03:45:13 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c0e4c593-b1a4-4684-a3f3-5b48fdd77d06 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "a264f3f7-b0ea-482a-a438-909f06a80726" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.927s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:14 user nova-compute[72081]: DEBUG nova.compute.manager [req-f38a63ff-e678-4fba-8b2d-d889ca0bcff8 req-6e031ed0-8e9f-4097-b7be-9e2d0f7e0efa service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Received event network-vif-plugged-c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:45:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f38a63ff-e678-4fba-8b2d-d889ca0bcff8 req-6e031ed0-8e9f-4097-b7be-9e2d0f7e0efa service nova] Acquiring lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f38a63ff-e678-4fba-8b2d-d889ca0bcff8 req-6e031ed0-8e9f-4097-b7be-9e2d0f7e0efa service nova] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f38a63ff-e678-4fba-8b2d-d889ca0bcff8 req-6e031ed0-8e9f-4097-b7be-9e2d0f7e0efa service nova] Lock "2d7f71ac-a671-4feb-8c2d-4cbcb84f7170-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:14 user nova-compute[72081]: DEBUG nova.compute.manager [req-f38a63ff-e678-4fba-8b2d-d889ca0bcff8 req-6e031ed0-8e9f-4097-b7be-9e2d0f7e0efa service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] No waiting events found dispatching network-vif-plugged-c904583a-ea96-45c9-ad62-d5e0cc68031f {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:45:14 user nova-compute[72081]: WARNING nova.compute.manager [req-f38a63ff-e678-4fba-8b2d-d889ca0bcff8 req-6e031ed0-8e9f-4097-b7be-9e2d0f7e0efa service nova] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Received unexpected event network-vif-plugged-c904583a-ea96-45c9-ad62-d5e0cc68031f for instance with vm_state deleted and task_state None.
Apr 12 03:45:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdg (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:45:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.158s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:45:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:45:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/614495f2-f108-4759-be60-bc6cc689c5b7/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/614495f2-f108-4759-be60-bc6cc689c5b7/disk.rescue --force-share --output=json" returned: 0 in 0.160s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/614495f2-f108-4759-be60-bc6cc689c5b7/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/614495f2-f108-4759-be60-bc6cc689c5b7/disk.rescue --force-share --output=json" returned: 0 in 0.155s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:45:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.171s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sde (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:45:16 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:45:16 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:45:16 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8021MB free_disk=25.93685531616211GB free_vcpus=5 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:45:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:16 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 2d0904c3-74e0-47bb-a832-758a7035b2f5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:45:16 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance bcd267d2-5b2c-4ed2-b634-0210fae94dfd actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:45:16 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 3d4407f4-e078-4777-a110-cb3f922be2e8 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:45:16 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 614495f2-f108-4759-be60-bc6cc689c5b7 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:45:16 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 606425bb-8968-41f9-b3a5-80b6fd188a67 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:45:16 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 48977eb2-ce43-4577-a1cb-340e35703dbd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 128, 'DISK_GB': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:45:16 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 93e45116-bb5c-40ca-9fc5-1c5c40ff687e actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:45:16 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance a264f3f7-b0ea-482a-a438-909f06a80726 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:45:16 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 8 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:45:16 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=1536MB phys_disk=40GB used_disk=5GB total_vcpus=12 used_vcpus=8 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:45:16 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:16 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:45:16 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:45:16 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:45:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.491s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:17 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:45:17 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:45:17 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:45:17 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Didn't find any instances for network info cache update. {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 12 03:45:17 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:45:17 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:45:17 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:45:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:20 user nova-compute[72081]: DEBUG nova.compute.manager [req-a6b8d93b-c3eb-4858-aacc-801d85cb7dc3 req-6ae99ad9-5209-4c01-b466-712fc9c6455a service nova] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Received event network-changed-e210f496-084e-443f-9c7f-684661441f6c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:45:20 user nova-compute[72081]: DEBUG nova.compute.manager [req-a6b8d93b-c3eb-4858-aacc-801d85cb7dc3 req-6ae99ad9-5209-4c01-b466-712fc9c6455a service nova] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Refreshing instance network info cache due to event network-changed-e210f496-084e-443f-9c7f-684661441f6c. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:45:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a6b8d93b-c3eb-4858-aacc-801d85cb7dc3 req-6ae99ad9-5209-4c01-b466-712fc9c6455a service nova] Acquiring lock "refresh_cache-93e45116-bb5c-40ca-9fc5-1c5c40ff687e" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:45:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a6b8d93b-c3eb-4858-aacc-801d85cb7dc3 req-6ae99ad9-5209-4c01-b466-712fc9c6455a service nova] Acquired lock "refresh_cache-93e45116-bb5c-40ca-9fc5-1c5c40ff687e" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:45:20 user nova-compute[72081]: DEBUG nova.network.neutron [req-a6b8d93b-c3eb-4858-aacc-801d85cb7dc3 req-6ae99ad9-5209-4c01-b466-712fc9c6455a service nova] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Refreshing network info cache for port e210f496-084e-443f-9c7f-684661441f6c {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:45:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "93e45116-bb5c-40ca-9fc5-1c5c40ff687e" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "93e45116-bb5c-40ca-9fc5-1c5c40ff687e" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "93e45116-bb5c-40ca-9fc5-1c5c40ff687e-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "93e45116-bb5c-40ca-9fc5-1c5c40ff687e-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "93e45116-bb5c-40ca-9fc5-1c5c40ff687e-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:20 user nova-compute[72081]: INFO nova.compute.manager [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Terminating instance
Apr 12 03:45:20 user nova-compute[72081]: DEBUG nova.compute.manager [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG nova.network.neutron [req-a6b8d93b-c3eb-4858-aacc-801d85cb7dc3 req-6ae99ad9-5209-4c01-b466-712fc9c6455a service nova] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Updated VIF entry in instance network info cache for port e210f496-084e-443f-9c7f-684661441f6c. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG nova.network.neutron [req-a6b8d93b-c3eb-4858-aacc-801d85cb7dc3 req-6ae99ad9-5209-4c01-b466-712fc9c6455a service nova] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Updating instance_info_cache with network_info: [{"id": "e210f496-084e-443f-9c7f-684661441f6c", "address": "fa:16:3e:9e:2a:c9", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.61", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape210f496-08", "ovs_interfaceid": "e210f496-084e-443f-9c7f-684661441f6c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a6b8d93b-c3eb-4858-aacc-801d85cb7dc3 req-6ae99ad9-5209-4c01-b466-712fc9c6455a service nova] Releasing lock "refresh_cache-93e45116-bb5c-40ca-9fc5-1c5c40ff687e" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:45:21 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Instance destroyed successfully.
Apr 12 03:45:21 user nova-compute[72081]: DEBUG nova.objects.instance [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lazy-loading 'resources' on Instance uuid 93e45116-bb5c-40ca-9fc5-1c5c40ff687e {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:43:27Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-565513575',display_name='tempest-TestVolumeBootPattern-server-565513575',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-565513575',id=32,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBASnyncVOhHKPzClYx3zwpDcJrieJkBwHOEHWlg2NFTJJdBa1/TdotUOLxC4yYnCsrHxIQy7r8qgZsehiquLQNSYVTnDsRU6DmQUCbMUYEphptEyPo7yf2/NkfI88nssBw==',key_name='tempest-TestVolumeBootPattern-54080024',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:43:40Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='003688353e2f4b36831c574ebaf65173',ramdisk_id='',reservation_id='r-48mm0c6c',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeBootPattern-235043461',owner_user_name='tempest-TestVolumeBootPattern-235043461-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:43:41Z,user_data=None,user_id='b00b348933224558aad1cb12921f100f',uuid=93e45116-bb5c-40ca-9fc5-1c5c40ff687e,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e210f496-084e-443f-9c7f-684661441f6c", "address": "fa:16:3e:9e:2a:c9", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape210f496-08", "ovs_interfaceid": "e210f496-084e-443f-9c7f-684661441f6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converting VIF {"id": "e210f496-084e-443f-9c7f-684661441f6c", "address": "fa:16:3e:9e:2a:c9", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape210f496-08", "ovs_interfaceid": "e210f496-084e-443f-9c7f-684661441f6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:9e:2a:c9,bridge_name='br-int',has_traffic_filtering=True,id=e210f496-084e-443f-9c7f-684661441f6c,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape210f496-08') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG os_vif [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:9e:2a:c9,bridge_name='br-int',has_traffic_filtering=True,id=e210f496-084e-443f-9c7f-684661441f6c,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape210f496-08') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tape210f496-08, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:21 user nova-compute[72081]: INFO os_vif [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:9e:2a:c9,bridge_name='br-int',has_traffic_filtering=True,id=e210f496-084e-443f-9c7f-684661441f6c,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape210f496-08')
Apr 12 03:45:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wmlvgqFk', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:45:21 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:45:21 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[62eab20e-6a0d-40c3-a0a0-9095b965e375]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:45:21 user nova-compute[72081]: No targets found.
Apr 12 03:45:21 user nova-compute[72081]: iSNS:
Apr 12 03:45:21 user nova-compute[72081]: No targets found.
Apr 12 03:45:21 user nova-compute[72081]: STATIC:
Apr 12 03:45:21 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27
Apr 12 03:45:21 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:45:21 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:45:21 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:45:21 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:45:21 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:45:21 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e
Apr 12 03:45:21 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:45:21 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:45:21 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068
Apr 12 03:45:21 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:45:21 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:45:21 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742
Apr 12 03:45:21 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:45:21 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:45:21 user nova-compute[72081]: FIRMWARE:
Apr 12 03:45:21 user nova-compute[72081]: No targets found.
Apr 12 03:45:21 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:45:21 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:45:21 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[efa8b8f6-ae39-45a6-b6f8-534cb40c7152]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[976df474-9a68-497a-9ad2-c1162f68e104]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:45:21 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:45:21 user nova-compute[72081]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:45:21 user nova-compute[72081]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:45:21 user nova-compute[72081]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:45:21 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:45:21 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:45:21 user nova-compute[72081]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:45:21 user nova-compute[72081]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:45:21 user nova-compute[72081]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:45:21 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fac11fbe0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27'): ({'sdd'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Removing single pathed devices sdd {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[37f7601a-d6d0-42f7-ba63-475871758f43]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.034s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4c6e4735-d914-47c8-b510-ad72167b1769]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Flushing IO for device /dev/sdd {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2bd1b16e-36f5-4771-92c6-2afdd6762ffa]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.050s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[26058da3-05e2-4bac-a98e-06a206b0a7bd]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] SCSI volumes sdd have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-1SCST_BIO3b0c3b20-a55db3597fce0c73', '/dev/disk/by-id/scsi-SSCST_BIO_Storage_3b0c3b20', '/dev/disk/by-id/scsi-26135356462333539') {{(pid=82083) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f47ce25a-4573-47e7-af12-aa25cf3afb88]: (4, None) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.029s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b82451b1-a7e8-4a79-837a-ff7b5d509352]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:45:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --logout" returned: 0 in 0.073s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:22 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5525dd86-36c3-47cd-b335-6baea8a0234d]: (4, ('Logging out of session [sid: 32, target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27, portal: 172.16.0.220,3260]\nLogout of [sid: 32, target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:22 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 32, target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27, portal: 172.16.0.220,3260]
Apr 12 03:45:22 user nova-compute[72081]: Logout of [sid: 32, target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27, portal: 172.16.0.220,3260] successful.
Apr 12 03:45:22 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:45:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.029s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:22 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[cbf2ef6b-87bc-4b9d-b7b1-8ef5ed93a24d]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:22 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:45:22 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.388s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:45:22 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] <== disconnect_volume: return (389ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:45:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:45:22 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Deleting instance files /opt/stack/data/nova/instances/93e45116-bb5c-40ca-9fc5-1c5c40ff687e_del
Apr 12 03:45:22 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Deletion of /opt/stack/data/nova/instances/93e45116-bb5c-40ca-9fc5-1c5c40ff687e_del complete
Apr 12 03:45:22 user nova-compute[72081]: INFO nova.compute.manager [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Took 1.25 seconds to destroy the instance on the hypervisor.
Apr 12 03:45:22 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:45:22 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:45:22 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:45:22 user nova-compute[72081]: DEBUG nova.compute.manager [req-2c3a7a8a-b307-4cde-9d26-2c0f98dda402 req-fbf436a7-b57c-4426-a570-10c5f1189609 service nova] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Received event network-vif-unplugged-e210f496-084e-443f-9c7f-684661441f6c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:45:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2c3a7a8a-b307-4cde-9d26-2c0f98dda402 req-fbf436a7-b57c-4426-a570-10c5f1189609 service nova] Acquiring lock "93e45116-bb5c-40ca-9fc5-1c5c40ff687e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2c3a7a8a-b307-4cde-9d26-2c0f98dda402 req-fbf436a7-b57c-4426-a570-10c5f1189609 service nova] Lock "93e45116-bb5c-40ca-9fc5-1c5c40ff687e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2c3a7a8a-b307-4cde-9d26-2c0f98dda402 req-fbf436a7-b57c-4426-a570-10c5f1189609 service nova] Lock "93e45116-bb5c-40ca-9fc5-1c5c40ff687e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:22 user nova-compute[72081]: DEBUG nova.compute.manager [req-2c3a7a8a-b307-4cde-9d26-2c0f98dda402 req-fbf436a7-b57c-4426-a570-10c5f1189609 service nova] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] No waiting events found dispatching network-vif-unplugged-e210f496-084e-443f-9c7f-684661441f6c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:45:22 user nova-compute[72081]: DEBUG nova.compute.manager [req-2c3a7a8a-b307-4cde-9d26-2c0f98dda402 req-fbf436a7-b57c-4426-a570-10c5f1189609 service nova] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Received event network-vif-unplugged-e210f496-084e-443f-9c7f-684661441f6c for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:45:22 user nova-compute[72081]: DEBUG nova.compute.manager [req-2c3a7a8a-b307-4cde-9d26-2c0f98dda402 req-fbf436a7-b57c-4426-a570-10c5f1189609 service nova] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Received event network-vif-plugged-e210f496-084e-443f-9c7f-684661441f6c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:45:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2c3a7a8a-b307-4cde-9d26-2c0f98dda402 req-fbf436a7-b57c-4426-a570-10c5f1189609 service nova] Acquiring lock "93e45116-bb5c-40ca-9fc5-1c5c40ff687e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2c3a7a8a-b307-4cde-9d26-2c0f98dda402 req-fbf436a7-b57c-4426-a570-10c5f1189609 service nova] Lock "93e45116-bb5c-40ca-9fc5-1c5c40ff687e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2c3a7a8a-b307-4cde-9d26-2c0f98dda402 req-fbf436a7-b57c-4426-a570-10c5f1189609 service nova] Lock "93e45116-bb5c-40ca-9fc5-1c5c40ff687e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:22 user nova-compute[72081]: DEBUG nova.compute.manager [req-2c3a7a8a-b307-4cde-9d26-2c0f98dda402 req-fbf436a7-b57c-4426-a570-10c5f1189609 service nova] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] No waiting events found dispatching network-vif-plugged-e210f496-084e-443f-9c7f-684661441f6c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:45:22 user nova-compute[72081]: WARNING nova.compute.manager [req-2c3a7a8a-b307-4cde-9d26-2c0f98dda402 req-fbf436a7-b57c-4426-a570-10c5f1189609 service nova] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Received unexpected event network-vif-plugged-e210f496-084e-443f-9c7f-684661441f6c for instance with vm_state active and task_state deleting.
Apr 12 03:45:22 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:45:23 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Took 0.84 seconds to deallocate network for instance.
Apr 12 03:45:23 user nova-compute[72081]: INFO nova.compute.manager [None req-621e9f3c-515f-4893-bfc1-ea3f92cd9ed7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Unrescuing
Apr 12 03:45:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-621e9f3c-515f-4893-bfc1-ea3f92cd9ed7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "refresh_cache-614495f2-f108-4759-be60-bc6cc689c5b7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:45:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-621e9f3c-515f-4893-bfc1-ea3f92cd9ed7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquired lock "refresh_cache-614495f2-f108-4759-be60-bc6cc689c5b7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:45:23 user nova-compute[72081]: DEBUG nova.network.neutron [None req-621e9f3c-515f-4893-bfc1-ea3f92cd9ed7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:45:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:24 user nova-compute[72081]: INFO nova.compute.manager [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Took 1.00 seconds to detach 1 volumes for instance.
Apr 12 03:45:24 user nova-compute[72081]: DEBUG nova.network.neutron [None req-621e9f3c-515f-4893-bfc1-ea3f92cd9ed7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Updating instance_info_cache with network_info: [{"id": "b9acaee6-a684-4827-bc91-1c6219782efe", "address": "fa:16:3e:d4:dc:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb9acaee6-a6", "ovs_interfaceid": "b9acaee6-a684-4827-bc91-1c6219782efe", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:45:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-621e9f3c-515f-4893-bfc1-ea3f92cd9ed7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Releasing lock "refresh_cache-614495f2-f108-4759-be60-bc6cc689c5b7" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:45:24 user nova-compute[72081]: DEBUG nova.objects.instance [None req-621e9f3c-515f-4893-bfc1-ea3f92cd9ed7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lazy-loading 'flavor' on Instance uuid 614495f2-f108-4759-be60-bc6cc689c5b7 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:45:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:24 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:45:24 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:45:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.572s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:24 user nova-compute[72081]: DEBUG nova.compute.manager [req-7cf3b245-98d9-43bd-9fad-758ceca79fb3 req-04a4e66b-7621-409a-88f6-01dae8f04733 service nova] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Received event network-vif-deleted-e210f496-084e-443f-9c7f-684661441f6c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:45:24 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Deleted allocations for instance 93e45116-bb5c-40ca-9fc5-1c5c40ff687e
Apr 12 03:45:24 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Instance destroyed successfully.
Apr 12 03:45:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-517fea11-011b-485f-8fd7-6ddb69d397cb tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "93e45116-bb5c-40ca-9fc5-1c5c40ff687e" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 4.087s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:25 user nova-compute[72081]: DEBUG nova.compute.manager [req-dd060dba-1504-4dc0-a7bf-70626637661a req-0164eadf-6532-4195-b049-840be8742c48 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received event network-vif-unplugged-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:45:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-dd060dba-1504-4dc0-a7bf-70626637661a req-0164eadf-6532-4195-b049-840be8742c48 service nova] Acquiring lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-dd060dba-1504-4dc0-a7bf-70626637661a req-0164eadf-6532-4195-b049-840be8742c48 service nova] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-dd060dba-1504-4dc0-a7bf-70626637661a req-0164eadf-6532-4195-b049-840be8742c48 service nova] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:25 user nova-compute[72081]: DEBUG nova.compute.manager [req-dd060dba-1504-4dc0-a7bf-70626637661a req-0164eadf-6532-4195-b049-840be8742c48 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] No waiting events found dispatching network-vif-unplugged-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:45:25 user nova-compute[72081]: WARNING nova.compute.manager [req-dd060dba-1504-4dc0-a7bf-70626637661a req-0164eadf-6532-4195-b049-840be8742c48 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received unexpected event network-vif-unplugged-b9acaee6-a684-4827-bc91-1c6219782efe for instance with vm_state rescued and task_state unrescuing.
Apr 12 03:45:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:27 user nova-compute[72081]: DEBUG nova.compute.manager [req-7182a920-ca27-4fd2-b4ef-4a293815b89d req-97ba58d0-319d-41f5-a7a4-81d21c6e7953 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received event network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:45:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7182a920-ca27-4fd2-b4ef-4a293815b89d req-97ba58d0-319d-41f5-a7a4-81d21c6e7953 service nova] Acquiring lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7182a920-ca27-4fd2-b4ef-4a293815b89d req-97ba58d0-319d-41f5-a7a4-81d21c6e7953 service nova] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7182a920-ca27-4fd2-b4ef-4a293815b89d req-97ba58d0-319d-41f5-a7a4-81d21c6e7953 service nova] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:27 user nova-compute[72081]: DEBUG nova.compute.manager [req-7182a920-ca27-4fd2-b4ef-4a293815b89d req-97ba58d0-319d-41f5-a7a4-81d21c6e7953 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] No waiting events found dispatching network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:45:27 user nova-compute[72081]: WARNING nova.compute.manager [req-7182a920-ca27-4fd2-b4ef-4a293815b89d req-97ba58d0-319d-41f5-a7a4-81d21c6e7953 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received unexpected event network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe for instance with vm_state rescued and task_state unrescuing.
Apr 12 03:45:27 user nova-compute[72081]: DEBUG nova.compute.manager [req-7182a920-ca27-4fd2-b4ef-4a293815b89d req-97ba58d0-319d-41f5-a7a4-81d21c6e7953 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received event network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:45:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7182a920-ca27-4fd2-b4ef-4a293815b89d req-97ba58d0-319d-41f5-a7a4-81d21c6e7953 service nova] Acquiring lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7182a920-ca27-4fd2-b4ef-4a293815b89d req-97ba58d0-319d-41f5-a7a4-81d21c6e7953 service nova] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7182a920-ca27-4fd2-b4ef-4a293815b89d req-97ba58d0-319d-41f5-a7a4-81d21c6e7953 service nova] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:27 user nova-compute[72081]: DEBUG nova.compute.manager [req-7182a920-ca27-4fd2-b4ef-4a293815b89d req-97ba58d0-319d-41f5-a7a4-81d21c6e7953 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] No waiting events found dispatching network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:45:27 user nova-compute[72081]: WARNING nova.compute.manager [req-7182a920-ca27-4fd2-b4ef-4a293815b89d req-97ba58d0-319d-41f5-a7a4-81d21c6e7953 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received unexpected event network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe for instance with vm_state rescued and task_state unrescuing.
Apr 12 03:45:27 user nova-compute[72081]: DEBUG nova.compute.manager [req-7182a920-ca27-4fd2-b4ef-4a293815b89d req-97ba58d0-319d-41f5-a7a4-81d21c6e7953 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received event network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:45:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7182a920-ca27-4fd2-b4ef-4a293815b89d req-97ba58d0-319d-41f5-a7a4-81d21c6e7953 service nova] Acquiring lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7182a920-ca27-4fd2-b4ef-4a293815b89d req-97ba58d0-319d-41f5-a7a4-81d21c6e7953 service nova] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7182a920-ca27-4fd2-b4ef-4a293815b89d req-97ba58d0-319d-41f5-a7a4-81d21c6e7953 service nova] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:27 user nova-compute[72081]: DEBUG nova.compute.manager [req-7182a920-ca27-4fd2-b4ef-4a293815b89d req-97ba58d0-319d-41f5-a7a4-81d21c6e7953 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] No waiting events found dispatching network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:45:27 user nova-compute[72081]: WARNING nova.compute.manager [req-7182a920-ca27-4fd2-b4ef-4a293815b89d req-97ba58d0-319d-41f5-a7a4-81d21c6e7953 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received unexpected event network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe for instance with vm_state rescued and task_state unrescuing.
Apr 12 03:45:27 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681271112.4570756, 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:45:27 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] VM Stopped (Lifecycle Event)
Apr 12 03:45:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-218c0e38-2066-4195-9b2a-8363fe5e15cb None None] [instance: 2d7f71ac-a671-4feb-8c2d-4cbcb84f7170] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:45:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:28 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Removed pending event for 614495f2-f108-4759-be60-bc6cc689c5b7 due to event {{(pid=72081) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 12 03:45:28 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271128.628885, 614495f2-f108-4759-be60-bc6cc689c5b7 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:45:28 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] VM Resumed (Lifecycle Event)
Apr 12 03:45:28 user nova-compute[72081]: DEBUG nova.compute.manager [None req-621e9f3c-515f-4893-bfc1-ea3f92cd9ed7 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:45:28 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:45:28 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:45:28 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] During sync_power_state the instance has a pending task (unrescuing). Skip.
Apr 12 03:45:28 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271128.6308951, 614495f2-f108-4759-be60-bc6cc689c5b7 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:45:28 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] VM Started (Lifecycle Event)
Apr 12 03:45:28 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:45:28 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Synchronizing instance power state after lifecycle event "Started"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:45:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:32 user nova-compute[72081]: DEBUG nova.compute.manager [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:45:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:32 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:45:32 user nova-compute[72081]: INFO nova.compute.claims [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Claim successful on node user
Apr 12 03:45:32 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:45:32 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:45:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.367s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:32 user nova-compute[72081]: DEBUG nova.compute.manager [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:45:32 user nova-compute[72081]: DEBUG nova.compute.manager [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:45:32 user nova-compute[72081]: DEBUG nova.network.neutron [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:45:32 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:45:32 user nova-compute[72081]: DEBUG nova.compute.manager [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:45:32 user nova-compute[72081]: DEBUG nova.policy [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '88c496d8e488434e833e67df25def576', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c426e52304c14ed9b0c1aed69691bc3d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:45:32 user nova-compute[72081]: DEBUG nova.compute.manager [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:45:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:45:32 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Creating image(s)
Apr 12 03:45:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "/opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "/opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "/opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.008s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.147s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.151s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk 1073741824" returned: 0 in 0.050s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.209s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.144s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:33 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Checking if we can resize image /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:45:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:33 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Cannot resize image /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:45:33 user nova-compute[72081]: DEBUG nova.objects.instance [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lazy-loading 'migration_context' on Instance uuid 7c2344ea-3ed4-40f5-b318-6fa8588b1fed {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:45:33 user nova-compute[72081]: DEBUG nova.network.neutron [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Successfully created port: c2ac7de8-155e-4d88-8534-5a5291bd18dc {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:45:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:45:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Ensure instance console log exists: /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:45:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.network.neutron [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Successfully updated port: c2ac7de8-155e-4d88-8534-5a5291bd18dc {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "refresh_cache-7c2344ea-3ed4-40f5-b318-6fa8588b1fed" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquired lock "refresh_cache-7c2344ea-3ed4-40f5-b318-6fa8588b1fed" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.network.neutron [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.network.neutron [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.compute.manager [req-11293f69-9d92-4aa0-8485-99b5ac9b6519 req-cf045d71-8c12-4551-8805-702624fa28a4 service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Received event network-changed-c2ac7de8-155e-4d88-8534-5a5291bd18dc {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.compute.manager [req-11293f69-9d92-4aa0-8485-99b5ac9b6519 req-cf045d71-8c12-4551-8805-702624fa28a4 service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Refreshing instance network info cache due to event network-changed-c2ac7de8-155e-4d88-8534-5a5291bd18dc. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-11293f69-9d92-4aa0-8485-99b5ac9b6519 req-cf045d71-8c12-4551-8805-702624fa28a4 service nova] Acquiring lock "refresh_cache-7c2344ea-3ed4-40f5-b318-6fa8588b1fed" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.network.neutron [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Updating instance_info_cache with network_info: [{"id": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "address": "fa:16:3e:6d:2b:fe", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2ac7de8-15", "ovs_interfaceid": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Releasing lock "refresh_cache-7c2344ea-3ed4-40f5-b318-6fa8588b1fed" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.compute.manager [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Instance network_info: |[{"id": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "address": "fa:16:3e:6d:2b:fe", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2ac7de8-15", "ovs_interfaceid": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-11293f69-9d92-4aa0-8485-99b5ac9b6519 req-cf045d71-8c12-4551-8805-702624fa28a4 service nova] Acquired lock "refresh_cache-7c2344ea-3ed4-40f5-b318-6fa8588b1fed" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.network.neutron [req-11293f69-9d92-4aa0-8485-99b5ac9b6519 req-cf045d71-8c12-4551-8805-702624fa28a4 service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Refreshing network info cache for port c2ac7de8-155e-4d88-8534-5a5291bd18dc {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Start _get_guest_xml network_info=[{"id": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "address": "fa:16:3e:6d:2b:fe", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2ac7de8-15", "ovs_interfaceid": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:45:34 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:45:34 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:45:31Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestStampPattern-server-1815563360',display_name='tempest-TestStampPattern-server-1815563360',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-teststamppattern-server-1815563360',id=34,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJxviRdZviloE/Y8vy1de4Nl8WuqpnNhf4wG8IpFYt8u0P1qgKwJzMzVj8wiWWgzOhRDflg4OM3oRDBzEH9LU9bTdJCvVUrt7G9IZqwugvdWPxD2R1lvN0sxiTneX8pngQ==',key_name='tempest-TestStampPattern-474439147',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='c426e52304c14ed9b0c1aed69691bc3d',ramdisk_id='',reservation_id='r-5vht163x',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestStampPattern-707517070',owner_user_name='tempest-TestStampPattern-707517070-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:45:33Z,user_data=None,user_id='88c496d8e488434e833e67df25def576',uuid=7c2344ea-3ed4-40f5-b318-6fa8588b1fed,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "address": "fa:16:3e:6d:2b:fe", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2ac7de8-15", "ovs_interfaceid": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Converting VIF {"id": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "address": "fa:16:3e:6d:2b:fe", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2ac7de8-15", "ovs_interfaceid": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6d:2b:fe,bridge_name='br-int',has_traffic_filtering=True,id=c2ac7de8-155e-4d88-8534-5a5291bd18dc,network=Network(8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc2ac7de8-15') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.objects.instance [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lazy-loading 'pci_devices' on Instance uuid 7c2344ea-3ed4-40f5-b318-6fa8588b1fed {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:45:34 user nova-compute[72081]:   <uuid>7c2344ea-3ed4-40f5-b318-6fa8588b1fed</uuid>
Apr 12 03:45:34 user nova-compute[72081]:   <name>instance-00000022</name>
Apr 12 03:45:34 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:45:34 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:45:34 user nova-compute[72081]:   <metadata>
Apr 12 03:45:34 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:45:34 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:45:34 user nova-compute[72081]:       <nova:name>tempest-TestStampPattern-server-1815563360</nova:name>
Apr 12 03:45:34 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:45:34</nova:creationTime>
Apr 12 03:45:34 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:45:34 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:45:34 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:45:34 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:45:34 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:45:34 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:45:34 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:45:34 user nova-compute[72081]:       <nova:owner>
Apr 12 03:45:34 user nova-compute[72081]:         <nova:user uuid="88c496d8e488434e833e67df25def576">tempest-TestStampPattern-707517070-project-member</nova:user>
Apr 12 03:45:34 user nova-compute[72081]:         <nova:project uuid="c426e52304c14ed9b0c1aed69691bc3d">tempest-TestStampPattern-707517070</nova:project>
Apr 12 03:45:34 user nova-compute[72081]:       </nova:owner>
Apr 12 03:45:34 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:45:34 user nova-compute[72081]:       <nova:ports>
Apr 12 03:45:34 user nova-compute[72081]:         <nova:port uuid="c2ac7de8-155e-4d88-8534-5a5291bd18dc">
Apr 12 03:45:34 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Apr 12 03:45:34 user nova-compute[72081]:         </nova:port>
Apr 12 03:45:34 user nova-compute[72081]:       </nova:ports>
Apr 12 03:45:34 user nova-compute[72081]:     </nova:instance>
Apr 12 03:45:34 user nova-compute[72081]:   </metadata>
Apr 12 03:45:34 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:45:34 user nova-compute[72081]:     <system>
Apr 12 03:45:34 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:45:34 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:45:34 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:45:34 user nova-compute[72081]:       <entry name="serial">7c2344ea-3ed4-40f5-b318-6fa8588b1fed</entry>
Apr 12 03:45:34 user nova-compute[72081]:       <entry name="uuid">7c2344ea-3ed4-40f5-b318-6fa8588b1fed</entry>
Apr 12 03:45:34 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:45:34 user nova-compute[72081]:     </system>
Apr 12 03:45:34 user nova-compute[72081]:   </sysinfo>
Apr 12 03:45:34 user nova-compute[72081]:   <os>
Apr 12 03:45:34 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:45:34 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:45:34 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:45:34 user nova-compute[72081]:   </os>
Apr 12 03:45:34 user nova-compute[72081]:   <features>
Apr 12 03:45:34 user nova-compute[72081]:     <acpi/>
Apr 12 03:45:34 user nova-compute[72081]:     <apic/>
Apr 12 03:45:34 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:45:34 user nova-compute[72081]:   </features>
Apr 12 03:45:34 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:45:34 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:45:34 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:45:34 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:45:34 user nova-compute[72081]:   </clock>
Apr 12 03:45:34 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:45:34 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:45:34 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:45:34 user nova-compute[72081]:   </cpu>
Apr 12 03:45:34 user nova-compute[72081]:   <devices>
Apr 12 03:45:34 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:45:34 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:45:34 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk"/>
Apr 12 03:45:34 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:45:34 user nova-compute[72081]:     </disk>
Apr 12 03:45:34 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:45:34 user nova-compute[72081]:       <mac address="fa:16:3e:6d:2b:fe"/>
Apr 12 03:45:34 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:45:34 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:45:34 user nova-compute[72081]:       <target dev="tapc2ac7de8-15"/>
Apr 12 03:45:34 user nova-compute[72081]:     </interface>
Apr 12 03:45:34 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:45:34 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/console.log" append="off"/>
Apr 12 03:45:34 user nova-compute[72081]:     </serial>
Apr 12 03:45:34 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:45:34 user nova-compute[72081]:     <video>
Apr 12 03:45:34 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:45:34 user nova-compute[72081]:     </video>
Apr 12 03:45:34 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:45:34 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:45:34 user nova-compute[72081]:     </rng>
Apr 12 03:45:34 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:45:34 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:45:34 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:45:34 user nova-compute[72081]:     </memballoon>
Apr 12 03:45:34 user nova-compute[72081]:   </devices>
Apr 12 03:45:34 user nova-compute[72081]: </domain>
Apr 12 03:45:34 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:45:31Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestStampPattern-server-1815563360',display_name='tempest-TestStampPattern-server-1815563360',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-teststamppattern-server-1815563360',id=34,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJxviRdZviloE/Y8vy1de4Nl8WuqpnNhf4wG8IpFYt8u0P1qgKwJzMzVj8wiWWgzOhRDflg4OM3oRDBzEH9LU9bTdJCvVUrt7G9IZqwugvdWPxD2R1lvN0sxiTneX8pngQ==',key_name='tempest-TestStampPattern-474439147',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='c426e52304c14ed9b0c1aed69691bc3d',ramdisk_id='',reservation_id='r-5vht163x',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestStampPattern-707517070',owner_user_name='tempest-TestStampPattern-707517070-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:45:33Z,user_data=None,user_id='88c496d8e488434e833e67df25def576',uuid=7c2344ea-3ed4-40f5-b318-6fa8588b1fed,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "address": "fa:16:3e:6d:2b:fe", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2ac7de8-15", "ovs_interfaceid": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Converting VIF {"id": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "address": "fa:16:3e:6d:2b:fe", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2ac7de8-15", "ovs_interfaceid": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6d:2b:fe,bridge_name='br-int',has_traffic_filtering=True,id=c2ac7de8-155e-4d88-8534-5a5291bd18dc,network=Network(8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc2ac7de8-15') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG os_vif [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:6d:2b:fe,bridge_name='br-int',has_traffic_filtering=True,id=c2ac7de8-155e-4d88-8534-5a5291bd18dc,network=Network(8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc2ac7de8-15') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapc2ac7de8-15, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapc2ac7de8-15, col_values=(('external_ids', {'iface-id': 'c2ac7de8-155e-4d88-8534-5a5291bd18dc', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:6d:2b:fe', 'vm-uuid': '7c2344ea-3ed4-40f5-b318-6fa8588b1fed'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:34 user nova-compute[72081]: INFO os_vif [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:6d:2b:fe,bridge_name='br-int',has_traffic_filtering=True,id=c2ac7de8-155e-4d88-8534-5a5291bd18dc,network=Network(8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc2ac7de8-15')
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] No VIF found with MAC fa:16:3e:6d:2b:fe, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.network.neutron [req-11293f69-9d92-4aa0-8485-99b5ac9b6519 req-cf045d71-8c12-4551-8805-702624fa28a4 service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Updated VIF entry in instance network info cache for port c2ac7de8-155e-4d88-8534-5a5291bd18dc. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG nova.network.neutron [req-11293f69-9d92-4aa0-8485-99b5ac9b6519 req-cf045d71-8c12-4551-8805-702624fa28a4 service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Updating instance_info_cache with network_info: [{"id": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "address": "fa:16:3e:6d:2b:fe", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2ac7de8-15", "ovs_interfaceid": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:45:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-11293f69-9d92-4aa0-8485-99b5ac9b6519 req-cf045d71-8c12-4551-8805-702624fa28a4 service nova] Releasing lock "refresh_cache-7c2344ea-3ed4-40f5-b318-6fa8588b1fed" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:45:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:36 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681271121.6644135, 93e45116-bb5c-40ca-9fc5-1c5c40ff687e => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:45:36 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] VM Stopped (Lifecycle Event)
Apr 12 03:45:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:36 user nova-compute[72081]: DEBUG nova.compute.manager [req-1f9e5059-95c6-4243-9e3d-54fa2c0dae1f req-df1c3a77-cee4-460a-a5c8-c0a532a8aaea service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Received event network-vif-plugged-c2ac7de8-155e-4d88-8534-5a5291bd18dc {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:45:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1f9e5059-95c6-4243-9e3d-54fa2c0dae1f req-df1c3a77-cee4-460a-a5c8-c0a532a8aaea service nova] Acquiring lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1f9e5059-95c6-4243-9e3d-54fa2c0dae1f req-df1c3a77-cee4-460a-a5c8-c0a532a8aaea service nova] Lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1f9e5059-95c6-4243-9e3d-54fa2c0dae1f req-df1c3a77-cee4-460a-a5c8-c0a532a8aaea service nova] Lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:36 user nova-compute[72081]: DEBUG nova.compute.manager [req-1f9e5059-95c6-4243-9e3d-54fa2c0dae1f req-df1c3a77-cee4-460a-a5c8-c0a532a8aaea service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] No waiting events found dispatching network-vif-plugged-c2ac7de8-155e-4d88-8534-5a5291bd18dc {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:45:36 user nova-compute[72081]: WARNING nova.compute.manager [req-1f9e5059-95c6-4243-9e3d-54fa2c0dae1f req-df1c3a77-cee4-460a-a5c8-c0a532a8aaea service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Received unexpected event network-vif-plugged-c2ac7de8-155e-4d88-8534-5a5291bd18dc for instance with vm_state building and task_state spawning.
Apr 12 03:45:36 user nova-compute[72081]: DEBUG nova.compute.manager [None req-7b2d496b-21d0-4304-814a-7dfcf9cc9d22 None None] [instance: 93e45116-bb5c-40ca-9fc5-1c5c40ff687e] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:45:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:38 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271138.098557, 7c2344ea-3ed4-40f5-b318-6fa8588b1fed => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:45:38 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] VM Resumed (Lifecycle Event)
Apr 12 03:45:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:45:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:45:38 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Instance spawned successfully.
Apr 12 03:45:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:45:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:45:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:45:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:45:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:45:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:45:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:45:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:45:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:45:38 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:45:38 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271138.101113, 7c2344ea-3ed4-40f5-b318-6fa8588b1fed => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:45:38 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] VM Started (Lifecycle Event)
Apr 12 03:45:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:45:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:45:38 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:45:38 user nova-compute[72081]: INFO nova.compute.manager [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Took 5.48 seconds to spawn the instance on the hypervisor.
Apr 12 03:45:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:45:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:38 user nova-compute[72081]: INFO nova.compute.manager [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Took 6.19 seconds to build instance.
Apr 12 03:45:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-62971632-69bc-46d5-a2aa-8d78583c202e tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.314s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-27d02620-661a-4295-964e-c033f0f6d18c req-9a385f83-9fde-430d-a67f-b22cf859da6c service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Received event network-vif-plugged-c2ac7de8-155e-4d88-8534-5a5291bd18dc {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:45:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-27d02620-661a-4295-964e-c033f0f6d18c req-9a385f83-9fde-430d-a67f-b22cf859da6c service nova] Acquiring lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-27d02620-661a-4295-964e-c033f0f6d18c req-9a385f83-9fde-430d-a67f-b22cf859da6c service nova] Lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-27d02620-661a-4295-964e-c033f0f6d18c req-9a385f83-9fde-430d-a67f-b22cf859da6c service nova] Lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-27d02620-661a-4295-964e-c033f0f6d18c req-9a385f83-9fde-430d-a67f-b22cf859da6c service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] No waiting events found dispatching network-vif-plugged-c2ac7de8-155e-4d88-8534-5a5291bd18dc {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:45:38 user nova-compute[72081]: WARNING nova.compute.manager [req-27d02620-661a-4295-964e-c033f0f6d18c req-9a385f83-9fde-430d-a67f-b22cf859da6c service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Received unexpected event network-vif-plugged-c2ac7de8-155e-4d88-8534-5a5291bd18dc for instance with vm_state active and task_state None.
Apr 12 03:45:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:54 user nova-compute[72081]: DEBUG nova.objects.instance [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'flavor' on Instance uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:45:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "refresh_cache-606425bb-8968-41f9-b3a5-80b6fd188a67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:45:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquired lock "refresh_cache-606425bb-8968-41f9-b3a5-80b6fd188a67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:45:54 user nova-compute[72081]: DEBUG nova.network.neutron [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:45:54 user nova-compute[72081]: DEBUG nova.objects.instance [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'info_cache' on Instance uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:45:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:54 user nova-compute[72081]: DEBUG nova.network.neutron [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Updating instance_info_cache with network_info: [{"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.205", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:45:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Releasing lock "refresh_cache-606425bb-8968-41f9-b3a5-80b6fd188a67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:45:54 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Instance destroyed successfully.
Apr 12 03:45:54 user nova-compute[72081]: DEBUG nova.objects.instance [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'resources' on Instance uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:45:54 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:42:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-2123178219',display_name='tempest-AttachVolumeTestJSON-server-2123178219',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-2123178219',id=31,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDsSrynzjRvdumscunYM7qq2LINQa1dLMdBCKJ6XwNU+wH30BpZ+UxgkL98cPRjcfl2uMCQ+6nM9+XV2FMWC/Uf6P+0R1p0Yeexa689yM1BKJkKUWaai92yYlAVXCdH/1Q==',key_name='tempest-keypair-611227490',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:42:09Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='a341e582192a47548df8627d82c96ea8',ramdisk_id='',reservation_id='r-qag42n01',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-617614045',owner_user_name='tempest-AttachVolumeTestJSON-617614045-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:44:04Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='872fbc3d59274cd480c17b4b7342499a',uuid=606425bb-8968-41f9-b3a5-80b6fd188a67,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.205", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:45:54 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Converting VIF {"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.205", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:45:54 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:85:88:9b,bridge_name='br-int',has_traffic_filtering=True,id=e16af111-f1d4-4f8c-a67a-c80f546c37b1,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16af111-f1') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:45:54 user nova-compute[72081]: DEBUG os_vif [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:85:88:9b,bridge_name='br-int',has_traffic_filtering=True,id=e16af111-f1d4-4f8c-a67a-c80f546c37b1,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16af111-f1') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:45:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tape16af111-f1, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:45:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:45:54 user nova-compute[72081]: INFO os_vif [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:85:88:9b,bridge_name='br-int',has_traffic_filtering=True,id=e16af111-f1d4-4f8c-a67a-c80f546c37b1,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16af111-f1')
Apr 12 03:45:54 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:45:54 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'VoqWQaSf', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:45:54 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:45:54 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:45:54 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:45:54 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b8c4b9ad-c18f-4e0a-b71d-5043de74c083]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:45:55 user nova-compute[72081]: No targets found.
Apr 12 03:45:55 user nova-compute[72081]: iSNS:
Apr 12 03:45:55 user nova-compute[72081]: No targets found.
Apr 12 03:45:55 user nova-compute[72081]: STATIC:
Apr 12 03:45:55 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:45:55 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:45:55 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:45:55 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e
Apr 12 03:45:55 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:45:55 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:45:55 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068
Apr 12 03:45:55 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:45:55 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:45:55 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742
Apr 12 03:45:55 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:45:55 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:45:55 user nova-compute[72081]: FIRMWARE:
Apr 12 03:45:55 user nova-compute[72081]: No targets found.
Apr 12 03:45:55 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:45:55 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:45:55 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[136b9116-36bd-4988-821b-49e6829f971b]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5be6ed9a-720f-4e92-95c5-a56cca9016b7]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:45:55 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:45:55 user nova-compute[72081]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:45:55 user nova-compute[72081]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:45:55 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:45:55 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:45:55 user nova-compute[72081]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:45:55 user nova-compute[72081]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:45:55 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3f9416fd00>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068'): ({'sde'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Removing single pathed devices sde {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f4aab1ac-7e30-419b-ad6d-0ae594f85fda]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9d0491d4-f4a5-4f7c-b9e3-32abbe6c4442]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Flushing IO for device /dev/sde {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[37602a8b-81f1-4d59-9234-0f847f9111fc]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.038s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a6541aa2-5648-4739-b23a-a3644352b51d]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] SCSI volumes sde have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.028s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[008a2daa-6aa0-4aea-996b-2af8161cb932]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --logout" returned: 0 in 0.076s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[db0fea22-1b0b-4e64-88f9-39f8e82d1154]: (4, ('Logging out of session [sid: 33, target: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068, portal: 172.16.0.220,3260]\nLogout of [sid: 33, target: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 33, target: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068, portal: 172.16.0.220,3260]
Apr 12 03:45:55 user nova-compute[72081]: Logout of [sid: 33, target: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068, portal: 172.16.0.220,3260] successful.
Apr 12 03:45:55 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5873dc6e-11b3-46f2-a21d-54a431fd777f]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.341s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] <== disconnect_volume: return (342ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Start _get_guest_xml network_info=[{"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.205", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vdb', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'VoqWQaSf', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde'}, 'status': 'reserved', 'instance': '606425bb-8968-41f9-b3a5-80b6fd188a67', 'attached_at': '', 'detached_at': '', 'volume_id': '188c90f2-90ef-4909-8917-56808fd9a068', 'serial': '188c90f2-90ef-4909-8917-56808fd9a068'}, 'disk_bus': 'virtio', 'delete_on_termination': False, 'boot_index': None, 'guest_format': None, 'device_type': 'disk', 'attachment_id': 'b6f0bcbc-0459-4eea-bd06-9911027035a9', 'volume_type': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG nova.objects.instance [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'numa_topology' on Instance uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:45:55 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:45:55 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:45:55 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG nova.objects.instance [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'vcpu_model' on Instance uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'VoqWQaSf', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:45:55 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:45:55 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:45:55 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[98d0b355-4b6a-4690-9b44-4989308f9da6]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:45:55 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[dfe14e3f-477a-4531-b21a-0c021740017d]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068] added
Apr 12 03:45:55 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5bc567e6-66de-42b4-9843-a7f215d8a02d]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e90ef7c1-f774-4a2c-8e34-17d74914256a]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v VoqWQaSf {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v VoqWQaSf" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[afa233cb-8bde-45ab-a187-dcce93f36161]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'VoqWQaSf'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[13322809-dfe9-4af5-ba8f-6fa14afafb35]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[83c6d69c-ae68-4428-8f36-9dc2a8e15951]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:45:55 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:45:55 user nova-compute[72081]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:45:55 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:45:55 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:45:55 user nova-compute[72081]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:45:55 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --login" returned: 0 in 0.064s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a62ce6b6-a4b0-4029-a0b4-4e6b8768ba33]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068, portal: 172.16.0.220,3260]
Apr 12 03:45:55 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068, portal: 172.16.0.220,3260] successful.
Apr 12 03:45:55 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[71738d11-2d9c-4c28-9dba-e04261e24b7a]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[cf0b5ddd-f86a-40a9-91b8-1dbbb19536c9]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:45:55 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:45:55 user nova-compute[72081]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:45:55 user nova-compute[72081]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:45:55 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:45:55 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:45:55 user nova-compute[72081]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:45:55 user nova-compute[72081]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:45:55 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.310s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] HCTL ('35', '-', '-', 0) found on session 36 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[63980d0c-3611-4cb0-a599-4fcb38e5e0f3]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Searching for a device in session 36 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:45:55 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'VoqWQaSf', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:45:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.345s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:45:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] <== connect_volume: return (1347ms) {'type': 'block', 'scsi_wwn': '23539616538316433', 'path': '/dev/sdd'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:45:56 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23539616538316433', 'path': '/dev/sdd'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:45:56 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:42:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-2123178219',display_name='tempest-AttachVolumeTestJSON-server-2123178219',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-2123178219',id=31,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDsSrynzjRvdumscunYM7qq2LINQa1dLMdBCKJ6XwNU+wH30BpZ+UxgkL98cPRjcfl2uMCQ+6nM9+XV2FMWC/Uf6P+0R1p0Yeexa689yM1BKJkKUWaai92yYlAVXCdH/1Q==',key_name='tempest-keypair-611227490',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:42:09Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='a341e582192a47548df8627d82c96ea8',ramdisk_id='',reservation_id='r-qag42n01',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-617614045',owner_user_name='tempest-AttachVolumeTestJSON-617614045-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:44:04Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='872fbc3d59274cd480c17b4b7342499a',uuid=606425bb-8968-41f9-b3a5-80b6fd188a67,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.205", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:45:56 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Converting VIF {"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.205", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:45:56 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:85:88:9b,bridge_name='br-int',has_traffic_filtering=True,id=e16af111-f1d4-4f8c-a67a-c80f546c37b1,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16af111-f1') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:45:56 user nova-compute[72081]: DEBUG nova.objects.instance [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'pci_devices' on Instance uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:45:56 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:45:56 user nova-compute[72081]:   <uuid>606425bb-8968-41f9-b3a5-80b6fd188a67</uuid>
Apr 12 03:45:56 user nova-compute[72081]:   <name>instance-0000001f</name>
Apr 12 03:45:56 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:45:56 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:45:56 user nova-compute[72081]:   <metadata>
Apr 12 03:45:56 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:45:56 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:45:56 user nova-compute[72081]:       <nova:name>tempest-AttachVolumeTestJSON-server-2123178219</nova:name>
Apr 12 03:45:56 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:45:55</nova:creationTime>
Apr 12 03:45:56 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:45:56 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:45:56 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:45:56 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:45:56 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:45:56 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:45:56 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:45:56 user nova-compute[72081]:       <nova:owner>
Apr 12 03:45:56 user nova-compute[72081]:         <nova:user uuid="872fbc3d59274cd480c17b4b7342499a">tempest-AttachVolumeTestJSON-617614045-project-member</nova:user>
Apr 12 03:45:56 user nova-compute[72081]:         <nova:project uuid="a341e582192a47548df8627d82c96ea8">tempest-AttachVolumeTestJSON-617614045</nova:project>
Apr 12 03:45:56 user nova-compute[72081]:       </nova:owner>
Apr 12 03:45:56 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:45:56 user nova-compute[72081]:       <nova:ports>
Apr 12 03:45:56 user nova-compute[72081]:         <nova:port uuid="e16af111-f1d4-4f8c-a67a-c80f546c37b1">
Apr 12 03:45:56 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Apr 12 03:45:56 user nova-compute[72081]:         </nova:port>
Apr 12 03:45:56 user nova-compute[72081]:       </nova:ports>
Apr 12 03:45:56 user nova-compute[72081]:     </nova:instance>
Apr 12 03:45:56 user nova-compute[72081]:   </metadata>
Apr 12 03:45:56 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:45:56 user nova-compute[72081]:     <system>
Apr 12 03:45:56 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:45:56 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:45:56 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:45:56 user nova-compute[72081]:       <entry name="serial">606425bb-8968-41f9-b3a5-80b6fd188a67</entry>
Apr 12 03:45:56 user nova-compute[72081]:       <entry name="uuid">606425bb-8968-41f9-b3a5-80b6fd188a67</entry>
Apr 12 03:45:56 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:45:56 user nova-compute[72081]:     </system>
Apr 12 03:45:56 user nova-compute[72081]:   </sysinfo>
Apr 12 03:45:56 user nova-compute[72081]:   <os>
Apr 12 03:45:56 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:45:56 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:45:56 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:45:56 user nova-compute[72081]:   </os>
Apr 12 03:45:56 user nova-compute[72081]:   <features>
Apr 12 03:45:56 user nova-compute[72081]:     <acpi/>
Apr 12 03:45:56 user nova-compute[72081]:     <apic/>
Apr 12 03:45:56 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:45:56 user nova-compute[72081]:   </features>
Apr 12 03:45:56 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:45:56 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:45:56 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:45:56 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:45:56 user nova-compute[72081]:   </clock>
Apr 12 03:45:56 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:45:56 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:45:56 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:45:56 user nova-compute[72081]:   </cpu>
Apr 12 03:45:56 user nova-compute[72081]:   <devices>
Apr 12 03:45:56 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:45:56 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:45:56 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk"/>
Apr 12 03:45:56 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:45:56 user nova-compute[72081]:     </disk>
Apr 12 03:45:56 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:45:56 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:45:56 user nova-compute[72081]:       <source dev="/dev/sdd"/>
Apr 12 03:45:56 user nova-compute[72081]:       <target dev="vdb" bus="virtio"/>
Apr 12 03:45:56 user nova-compute[72081]:       <serial>188c90f2-90ef-4909-8917-56808fd9a068</serial>
Apr 12 03:45:56 user nova-compute[72081]:     </disk>
Apr 12 03:45:56 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:45:56 user nova-compute[72081]:       <mac address="fa:16:3e:85:88:9b"/>
Apr 12 03:45:56 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:45:56 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:45:56 user nova-compute[72081]:       <target dev="tape16af111-f1"/>
Apr 12 03:45:56 user nova-compute[72081]:     </interface>
Apr 12 03:45:56 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:45:56 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/console.log" append="off"/>
Apr 12 03:45:56 user nova-compute[72081]:     </serial>
Apr 12 03:45:56 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:45:56 user nova-compute[72081]:     <video>
Apr 12 03:45:56 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:45:56 user nova-compute[72081]:     </video>
Apr 12 03:45:56 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:45:56 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:45:56 user nova-compute[72081]:     </rng>
Apr 12 03:45:56 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:45:56 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:45:56 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:45:56 user nova-compute[72081]:     </memballoon>
Apr 12 03:45:56 user nova-compute[72081]:   </devices>
Apr 12 03:45:56 user nova-compute[72081]: </domain>
Apr 12 03:45:56 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:45:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:57 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] skipping disk /dev/sdd (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:45:57 user nova-compute[72081]: DEBUG nova.objects.instance [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'trusted_certs' on Instance uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:45:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.140s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:57 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Checking if we can resize image /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:45:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:45:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:45:57 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Cannot resize image /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:45:57 user nova-compute[72081]: DEBUG nova.objects.instance [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'migration_context' on Instance uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:45:57 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:42:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-2123178219',display_name='tempest-AttachVolumeTestJSON-server-2123178219',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-2123178219',id=31,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDsSrynzjRvdumscunYM7qq2LINQa1dLMdBCKJ6XwNU+wH30BpZ+UxgkL98cPRjcfl2uMCQ+6nM9+XV2FMWC/Uf6P+0R1p0Yeexa689yM1BKJkKUWaai92yYlAVXCdH/1Q==',key_name='tempest-keypair-611227490',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:42:09Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=<?>,power_state=4,progress=0,project_id='a341e582192a47548df8627d82c96ea8',ramdisk_id='',reservation_id='r-qag42n01',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-617614045',owner_user_name='tempest-AttachVolumeTestJSON-617614045-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:44:04Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='872fbc3d59274cd480c17b4b7342499a',uuid=606425bb-8968-41f9-b3a5-80b6fd188a67,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.205", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:45:57 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Converting VIF {"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.205", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:45:57 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:85:88:9b,bridge_name='br-int',has_traffic_filtering=True,id=e16af111-f1d4-4f8c-a67a-c80f546c37b1,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16af111-f1') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:45:57 user nova-compute[72081]: DEBUG os_vif [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:85:88:9b,bridge_name='br-int',has_traffic_filtering=True,id=e16af111-f1d4-4f8c-a67a-c80f546c37b1,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16af111-f1') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:45:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:45:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:45:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tape16af111-f1, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:45:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tape16af111-f1, col_values=(('external_ids', {'iface-id': 'e16af111-f1d4-4f8c-a67a-c80f546c37b1', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:85:88:9b', 'vm-uuid': '606425bb-8968-41f9-b3a5-80b6fd188a67'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:45:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:45:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:57 user nova-compute[72081]: INFO os_vif [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:85:88:9b,bridge_name='br-int',has_traffic_filtering=True,id=e16af111-f1d4-4f8c-a67a-c80f546c37b1,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16af111-f1')
Apr 12 03:45:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-05baf7db-8c51-4227-b069-25d7de4cab6a req-f3b36553-e3af-430f-823f-10f20f170cfe service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:45:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-05baf7db-8c51-4227-b069-25d7de4cab6a req-f3b36553-e3af-430f-823f-10f20f170cfe service nova] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:45:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-05baf7db-8c51-4227-b069-25d7de4cab6a req-f3b36553-e3af-430f-823f-10f20f170cfe service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:45:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-05baf7db-8c51-4227-b069-25d7de4cab6a req-f3b36553-e3af-430f-823f-10f20f170cfe service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:45:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-05baf7db-8c51-4227-b069-25d7de4cab6a req-f3b36553-e3af-430f-823f-10f20f170cfe service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] No waiting events found dispatching network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:45:58 user nova-compute[72081]: WARNING nova.compute.manager [req-05baf7db-8c51-4227-b069-25d7de4cab6a req-f3b36553-e3af-430f-823f-10f20f170cfe service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received unexpected event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 for instance with vm_state stopped and task_state powering-on.
Apr 12 03:45:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:45:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:00 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271160.868638, 606425bb-8968-41f9-b3a5-80b6fd188a67 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:46:00 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] VM Resumed (Lifecycle Event)
Apr 12 03:46:00 user nova-compute[72081]: DEBUG nova.compute.manager [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:46:00 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Instance rebooted successfully.
Apr 12 03:46:00 user nova-compute[72081]: DEBUG nova.compute.manager [None req-74104a0f-044c-4918-ba9a-7bb3fbf0c507 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:46:00 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:46:00 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: stopped, current task_state: powering-on, current DB power_state: 4, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:46:00 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] During sync_power_state the instance has a pending task (powering-on). Skip.
Apr 12 03:46:00 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271160.8739617, 606425bb-8968-41f9-b3a5-80b6fd188a67 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:46:00 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] VM Started (Lifecycle Event)
Apr 12 03:46:00 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:46:00 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:46:01 user nova-compute[72081]: DEBUG nova.compute.manager [req-0df2cf4a-f23f-4124-8b26-92215363b682 req-1aee33d2-51ed-449e-8314-a43ab1c211f7 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:46:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0df2cf4a-f23f-4124-8b26-92215363b682 req-1aee33d2-51ed-449e-8314-a43ab1c211f7 service nova] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:46:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0df2cf4a-f23f-4124-8b26-92215363b682 req-1aee33d2-51ed-449e-8314-a43ab1c211f7 service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:46:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0df2cf4a-f23f-4124-8b26-92215363b682 req-1aee33d2-51ed-449e-8314-a43ab1c211f7 service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:46:01 user nova-compute[72081]: DEBUG nova.compute.manager [req-0df2cf4a-f23f-4124-8b26-92215363b682 req-1aee33d2-51ed-449e-8314-a43ab1c211f7 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] No waiting events found dispatching network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:46:01 user nova-compute[72081]: WARNING nova.compute.manager [req-0df2cf4a-f23f-4124-8b26-92215363b682 req-1aee33d2-51ed-449e-8314-a43ab1c211f7 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received unexpected event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 for instance with vm_state active and task_state None.
Apr 12 03:46:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:06 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:46:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:07 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:46:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Cleaning up deleted instances with incomplete migration  {{(pid=72081) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 12 03:46:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:11 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:46:11 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:46:11 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:46:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:12 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:46:12 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Cleaning up deleted instances {{(pid=72081) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 12 03:46:12 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] There are 0 instances to clean {{(pid=72081) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 12 03:46:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "12351dbd-0f2e-40a6-a10a-f40f5b6250ef" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:46:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "12351dbd-0f2e-40a6-a10a-f40f5b6250ef" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:46:13 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:46:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:46:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:46:13 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:46:13 user nova-compute[72081]: INFO nova.compute.claims [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Claim successful on node user
Apr 12 03:46:13 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Refreshing inventories for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 12 03:46:13 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Updating ProviderTree inventory for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 12 03:46:13 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Updating inventory in ProviderTree for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 12 03:46:13 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Refreshing aggregate associations for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6, aggregates: None {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 12 03:46:13 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Refreshing trait associations for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6, traits: COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_USB,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_RESCUE_BFV,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE41,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_STORAGE_BUS_FDC,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_STORAGE_BUS_IDE,COMPUTE_TRUSTED_CERTS,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_GRAPHICS_MODEL_CIRRUS,HW_CPU_X86_SSSE3,HW_CPU_X86_SSE,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_EXTEND,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_STORAGE_BUS_SATA,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_GRAPHICS_MODEL_QXL,HW_CPU_X86_SSE42,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_ACCELERATORS,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,HW_CPU_X86_MMX,COMPUTE_NODE,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_DEVICE_TAGGING,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_VIOMMU_MODEL_AUTO {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 12 03:46:13 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:46:13 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:46:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.539s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:46:13 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:46:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:46:14 user nova-compute[72081]: DEBUG nova.network.neutron [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:46:14 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:46:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:46:14 user nova-compute[72081]: INFO nova.virt.block_device [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Booting with volume ac69f59b-6bc4-4f63-bbf6-2952889fcf27 at /dev/vda
Apr 12 03:46:14 user nova-compute[72081]: DEBUG nova.policy [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'b00b348933224558aad1cb12921f100f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '003688353e2f4b36831c574ebaf65173', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:46:14 user nova-compute[72081]: DEBUG os_brick.utils [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:46:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[db20cfaa-ca3f-4d3f-9344-c1ff2c32797c]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[abc985b8-8a35-4b73-8583-9a4bfbf523c9]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.058s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ceaec96b-8d86-4f26-be5d-b99d6d38570d]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7007d5dc-30cf-44a3-95f7-25085d6853a8]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:46:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:46:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:46:14 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:46:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1012e542-75a4-4591-b72c-0de31820215e]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:46:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:46:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:46:14 user nova-compute[72081]: DEBUG os_brick.utils [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] <== get_connector_properties: return (168ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:46:14 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Updating existing volume attachment record: b0cf175a-6d6e-40d3-b832-7a1b28c3f672 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:46:14 user nova-compute[72081]: DEBUG nova.network.neutron [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Successfully created port: e1586e63-ee44-4717-ad47-886b1c3f01d0 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:46:15 user nova-compute[72081]: DEBUG nova.network.neutron [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Successfully updated port: e1586e63-ee44-4717-ad47-886b1c3f01d0 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:46:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "refresh_cache-12351dbd-0f2e-40a6-a10a-f40f5b6250ef" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:46:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquired lock "refresh_cache-12351dbd-0f2e-40a6-a10a-f40f5b6250ef" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:46:15 user nova-compute[72081]: DEBUG nova.network.neutron [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:46:15 user nova-compute[72081]: DEBUG nova.compute.manager [req-505e405c-ac38-466f-883b-c61e7297ceb4 req-f9d5c9a9-cd08-4035-9108-ba1e8102132f service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Received event network-changed-e1586e63-ee44-4717-ad47-886b1c3f01d0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:46:15 user nova-compute[72081]: DEBUG nova.compute.manager [req-505e405c-ac38-466f-883b-c61e7297ceb4 req-f9d5c9a9-cd08-4035-9108-ba1e8102132f service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Refreshing instance network info cache due to event network-changed-e1586e63-ee44-4717-ad47-886b1c3f01d0. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:46:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-505e405c-ac38-466f-883b-c61e7297ceb4 req-f9d5c9a9-cd08-4035-9108-ba1e8102132f service nova] Acquiring lock "refresh_cache-12351dbd-0f2e-40a6-a10a-f40f5b6250ef" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:46:15 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:46:15 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:46:15 user nova-compute[72081]: DEBUG nova.network.neutron [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:46:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:46:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:46:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:46:15 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:46:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:16 user nova-compute[72081]: DEBUG nova.network.neutron [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Updating instance_info_cache with network_info: [{"id": "e1586e63-ee44-4717-ad47-886b1c3f01d0", "address": "fa:16:3e:eb:02:66", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1586e63-ee", "ovs_interfaceid": "e1586e63-ee44-4717-ad47-886b1c3f01d0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:46:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Releasing lock "refresh_cache-12351dbd-0f2e-40a6-a10a-f40f5b6250ef" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:46:16 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Instance network_info: |[{"id": "e1586e63-ee44-4717-ad47-886b1c3f01d0", "address": "fa:16:3e:eb:02:66", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1586e63-ee", "ovs_interfaceid": "e1586e63-ee44-4717-ad47-886b1c3f01d0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:46:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-505e405c-ac38-466f-883b-c61e7297ceb4 req-f9d5c9a9-cd08-4035-9108-ba1e8102132f service nova] Acquired lock "refresh_cache-12351dbd-0f2e-40a6-a10a-f40f5b6250ef" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:46:16 user nova-compute[72081]: DEBUG nova.network.neutron [req-505e405c-ac38-466f-883b-c61e7297ceb4 req-f9d5c9a9-cd08-4035-9108-ba1e8102132f service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Refreshing network info cache for port e1586e63-ee44-4717-ad47-886b1c3f01d0 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:46:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:16 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdg (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:46:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:16 user nova-compute[72081]: DEBUG nova.network.neutron [req-505e405c-ac38-466f-883b-c61e7297ceb4 req-f9d5c9a9-cd08-4035-9108-ba1e8102132f service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Updated VIF entry in instance network info cache for port e1586e63-ee44-4717-ad47-886b1c3f01d0. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:46:16 user nova-compute[72081]: DEBUG nova.network.neutron [req-505e405c-ac38-466f-883b-c61e7297ceb4 req-f9d5c9a9-cd08-4035-9108-ba1e8102132f service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Updating instance_info_cache with network_info: [{"id": "e1586e63-ee44-4717-ad47-886b1c3f01d0", "address": "fa:16:3e:eb:02:66", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1586e63-ee", "ovs_interfaceid": "e1586e63-ee44-4717-ad47-886b1c3f01d0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:46:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-505e405c-ac38-466f-883b-c61e7297ceb4 req-f9d5c9a9-cd08-4035-9108-ba1e8102132f service nova] Releasing lock "refresh_cache-12351dbd-0f2e-40a6-a10a-f40f5b6250ef" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:46:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:46:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdd (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:46:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:46:18 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:46:18 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=7972MB free_disk=25.921920776367188GB free_vcpus=4 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 2d0904c3-74e0-47bb-a832-758a7035b2f5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance bcd267d2-5b2c-4ed2-b634-0210fae94dfd actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 3d4407f4-e078-4777-a110-cb3f922be2e8 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 614495f2-f108-4759-be60-bc6cc689c5b7 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 606425bb-8968-41f9-b3a5-80b6fd188a67 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 48977eb2-ce43-4577-a1cb-340e35703dbd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 128, 'DISK_GB': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance a264f3f7-b0ea-482a-a438-909f06a80726 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 7c2344ea-3ed4-40f5-b318-6fa8588b1fed actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 12351dbd-0f2e-40a6-a10a-f40f5b6250ef actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=1664MB phys_disk=40GB used_disk=6GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.414s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Rebuilding the list of instances to heal {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Skipping network cache update for instance because it is Building. {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9805}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-2d0904c3-74e0-47bb-a832-758a7035b2f5" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-2d0904c3-74e0-47bb-a832-758a7035b2f5" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lazy-loading 'info_cache' on Instance uuid 2d0904c3-74e0-47bb-a832-758a7035b2f5 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:46:18 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Creating image(s)
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Did not create local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Ensure instance console log exists: /opt/stack/data/nova/instances/12351dbd-0f2e-40a6-a10a-f40f5b6250ef/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Start _get_guest_xml network_info=[{"id": "e1586e63-ee44-4717-ad47-886b1c3f01d0", "address": "fa:16:3e:eb:02:66", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1586e63-ee", "ovs_interfaceid": "e1586e63-ee44-4717-ad47-886b1c3f01d0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wmlvgqFk', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '12351dbd-0f2e-40a6-a10a-f40f5b6250ef', 'attached_at': '', 'detached_at': '', 'volume_id': 'ac69f59b-6bc4-4f63-bbf6-2952889fcf27', 'serial': 'ac69f59b-6bc4-4f63-bbf6-2952889fcf27'}, 'disk_bus': 'virtio', 'delete_on_termination': False, 'boot_index': 0, 'guest_format': None, 'device_type': 'disk', 'attachment_id': 'b0cf175a-6d6e-40d3-b832-7a1b28c3f672', 'volume_type': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:46:18 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:46:18 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wmlvgqFk', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:46:18 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:46:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:46:18 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:46:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:46:18 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e52fcc3e-154c-4d73-b1f2-ffe0b2a93a82]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:46:18 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[802e1134-1488-4da7-a8a9-c37e6927a382]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27] added
Apr 12 03:46:18 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c86420d9-0e56-40ae-aeb8-cead470405a0]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[94d2e0e5-7ba8-4507-b654-45116dfbd7c4]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v wmlvgqFk {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v wmlvgqFk" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[722e877c-ec6d-4514-862b-4b791e57e187]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'wmlvgqFk'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[87baef78-80af-49ee-914b-492c8bbea1d0]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:46:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b3e1dc96-4529-4a57-ae73-f685749a82dd]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:46:19 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:46:19 user nova-compute[72081]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:46:19 user nova-compute[72081]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:46:19 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:46:19 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:46:19 user nova-compute[72081]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:46:19 user nova-compute[72081]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:46:19 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --login" returned: 0 in 0.069s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6a830678-27e2-47f5-b716-c5f51a30f4af]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27, portal: 172.16.0.220,3260]
Apr 12 03:46:19 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27, portal: 172.16.0.220,3260] successful.
Apr 12 03:46:19 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6cef08f2-5a1a-48cd-8f75-daf72a74de26]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[858e0bde-89da-415c-87eb-2b2ea0d233d1]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:46:19 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:46:19 user nova-compute[72081]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:46:19 user nova-compute[72081]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:46:19 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:46:19 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:46:19 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:46:19 user nova-compute[72081]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:46:19 user nova-compute[72081]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:46:19 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:46:19 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.296s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] HCTL ('36', '-', '-', 0) found on session 37 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[05d01ce5-fb84-45f3-9fbd-fb1187c54dd8]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Searching for a device in session 37 and hctl ['36', '*', '*', 0] yield: None {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Updating instance_info_cache with network_info: [{"id": "dce84390-a2f8-4047-b513-eec5845a2399", "address": "fa:16:3e:6e:2e:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdce84390-a2", "ovs_interfaceid": "dce84390-a2f8-4047-b513-eec5845a2399", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-2d0904c3-74e0-47bb-a832-758a7035b2f5" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:46:19 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:46:20 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Searching for a device in session 37 and hctl ['36', '*', '*', 0] yield: sde {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:46:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wmlvgqFk', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:46:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.328s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:46:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] <== connect_volume: return (1330ms) {'type': 'block', 'scsi_wwn': '26135356462333539', 'path': '/dev/sde'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:46:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26135356462333539', 'path': '/dev/sde'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:46:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:46:12Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1526891958',display_name='tempest-TestVolumeBootPattern-server-1526891958',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1526891958',id=35,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBASnyncVOhHKPzClYx3zwpDcJrieJkBwHOEHWlg2NFTJJdBa1/TdotUOLxC4yYnCsrHxIQy7r8qgZsehiquLQNSYVTnDsRU6DmQUCbMUYEphptEyPo7yf2/NkfI88nssBw==',key_name='tempest-TestVolumeBootPattern-54080024',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='003688353e2f4b36831c574ebaf65173',ramdisk_id='',reservation_id='r-q3m6prjl',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-235043461',owner_user_name='tempest-TestVolumeBootPattern-235043461-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:46:14Z,user_data=None,user_id='b00b348933224558aad1cb12921f100f',uuid=12351dbd-0f2e-40a6-a10a-f40f5b6250ef,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e1586e63-ee44-4717-ad47-886b1c3f01d0", "address": "fa:16:3e:eb:02:66", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1586e63-ee", "ovs_interfaceid": "e1586e63-ee44-4717-ad47-886b1c3f01d0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:46:20 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converting VIF {"id": "e1586e63-ee44-4717-ad47-886b1c3f01d0", "address": "fa:16:3e:eb:02:66", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1586e63-ee", "ovs_interfaceid": "e1586e63-ee44-4717-ad47-886b1c3f01d0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:46:20 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:eb:02:66,bridge_name='br-int',has_traffic_filtering=True,id=e1586e63-ee44-4717-ad47-886b1c3f01d0,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape1586e63-ee') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:46:20 user nova-compute[72081]: DEBUG nova.objects.instance [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lazy-loading 'pci_devices' on Instance uuid 12351dbd-0f2e-40a6-a10a-f40f5b6250ef {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:46:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:46:20 user nova-compute[72081]:   <uuid>12351dbd-0f2e-40a6-a10a-f40f5b6250ef</uuid>
Apr 12 03:46:20 user nova-compute[72081]:   <name>instance-00000023</name>
Apr 12 03:46:20 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:46:20 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:46:20 user nova-compute[72081]:   <metadata>
Apr 12 03:46:20 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:46:20 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:46:20 user nova-compute[72081]:       <nova:name>tempest-TestVolumeBootPattern-server-1526891958</nova:name>
Apr 12 03:46:20 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:46:18</nova:creationTime>
Apr 12 03:46:20 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:46:20 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:46:20 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:46:20 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:46:20 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:46:20 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:46:20 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:46:20 user nova-compute[72081]:       <nova:owner>
Apr 12 03:46:20 user nova-compute[72081]:         <nova:user uuid="b00b348933224558aad1cb12921f100f">tempest-TestVolumeBootPattern-235043461-project-member</nova:user>
Apr 12 03:46:20 user nova-compute[72081]:         <nova:project uuid="003688353e2f4b36831c574ebaf65173">tempest-TestVolumeBootPattern-235043461</nova:project>
Apr 12 03:46:20 user nova-compute[72081]:       </nova:owner>
Apr 12 03:46:20 user nova-compute[72081]:       <nova:ports>
Apr 12 03:46:20 user nova-compute[72081]:         <nova:port uuid="e1586e63-ee44-4717-ad47-886b1c3f01d0">
Apr 12 03:46:20 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 12 03:46:20 user nova-compute[72081]:         </nova:port>
Apr 12 03:46:20 user nova-compute[72081]:       </nova:ports>
Apr 12 03:46:20 user nova-compute[72081]:     </nova:instance>
Apr 12 03:46:20 user nova-compute[72081]:   </metadata>
Apr 12 03:46:20 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:46:20 user nova-compute[72081]:     <system>
Apr 12 03:46:20 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:46:20 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:46:20 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:46:20 user nova-compute[72081]:       <entry name="serial">12351dbd-0f2e-40a6-a10a-f40f5b6250ef</entry>
Apr 12 03:46:20 user nova-compute[72081]:       <entry name="uuid">12351dbd-0f2e-40a6-a10a-f40f5b6250ef</entry>
Apr 12 03:46:20 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:46:20 user nova-compute[72081]:     </system>
Apr 12 03:46:20 user nova-compute[72081]:   </sysinfo>
Apr 12 03:46:20 user nova-compute[72081]:   <os>
Apr 12 03:46:20 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:46:20 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:46:20 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:46:20 user nova-compute[72081]:   </os>
Apr 12 03:46:20 user nova-compute[72081]:   <features>
Apr 12 03:46:20 user nova-compute[72081]:     <acpi/>
Apr 12 03:46:20 user nova-compute[72081]:     <apic/>
Apr 12 03:46:20 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:46:20 user nova-compute[72081]:   </features>
Apr 12 03:46:20 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:46:20 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:46:20 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:46:20 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:46:20 user nova-compute[72081]:   </clock>
Apr 12 03:46:20 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:46:20 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:46:20 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:46:20 user nova-compute[72081]:   </cpu>
Apr 12 03:46:20 user nova-compute[72081]:   <devices>
Apr 12 03:46:20 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:46:20 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:46:20 user nova-compute[72081]:       <source dev="/dev/sde"/>
Apr 12 03:46:20 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:46:20 user nova-compute[72081]:       <serial>ac69f59b-6bc4-4f63-bbf6-2952889fcf27</serial>
Apr 12 03:46:20 user nova-compute[72081]:     </disk>
Apr 12 03:46:20 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:46:20 user nova-compute[72081]:       <mac address="fa:16:3e:eb:02:66"/>
Apr 12 03:46:20 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:46:20 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:46:20 user nova-compute[72081]:       <target dev="tape1586e63-ee"/>
Apr 12 03:46:20 user nova-compute[72081]:     </interface>
Apr 12 03:46:20 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:46:20 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/12351dbd-0f2e-40a6-a10a-f40f5b6250ef/console.log" append="off"/>
Apr 12 03:46:20 user nova-compute[72081]:     </serial>
Apr 12 03:46:20 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:46:20 user nova-compute[72081]:     <video>
Apr 12 03:46:20 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:46:20 user nova-compute[72081]:     </video>
Apr 12 03:46:20 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:46:20 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:46:20 user nova-compute[72081]:     </rng>
Apr 12 03:46:20 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:46:20 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:46:20 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:46:20 user nova-compute[72081]:     </memballoon>
Apr 12 03:46:20 user nova-compute[72081]:   </devices>
Apr 12 03:46:20 user nova-compute[72081]: </domain>
Apr 12 03:46:20 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:46:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:46:12Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1526891958',display_name='tempest-TestVolumeBootPattern-server-1526891958',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1526891958',id=35,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBASnyncVOhHKPzClYx3zwpDcJrieJkBwHOEHWlg2NFTJJdBa1/TdotUOLxC4yYnCsrHxIQy7r8qgZsehiquLQNSYVTnDsRU6DmQUCbMUYEphptEyPo7yf2/NkfI88nssBw==',key_name='tempest-TestVolumeBootPattern-54080024',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='003688353e2f4b36831c574ebaf65173',ramdisk_id='',reservation_id='r-q3m6prjl',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-235043461',owner_user_name='tempest-TestVolumeBootPattern-235043461-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:46:14Z,user_data=None,user_id='b00b348933224558aad1cb12921f100f',uuid=12351dbd-0f2e-40a6-a10a-f40f5b6250ef,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e1586e63-ee44-4717-ad47-886b1c3f01d0", "address": "fa:16:3e:eb:02:66", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1586e63-ee", "ovs_interfaceid": "e1586e63-ee44-4717-ad47-886b1c3f01d0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:46:20 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converting VIF {"id": "e1586e63-ee44-4717-ad47-886b1c3f01d0", "address": "fa:16:3e:eb:02:66", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1586e63-ee", "ovs_interfaceid": "e1586e63-ee44-4717-ad47-886b1c3f01d0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:46:20 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:eb:02:66,bridge_name='br-int',has_traffic_filtering=True,id=e1586e63-ee44-4717-ad47-886b1c3f01d0,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape1586e63-ee') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:46:20 user nova-compute[72081]: DEBUG os_vif [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:eb:02:66,bridge_name='br-int',has_traffic_filtering=True,id=e1586e63-ee44-4717-ad47-886b1c3f01d0,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape1586e63-ee') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:46:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:46:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:46:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tape1586e63-ee, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:46:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tape1586e63-ee, col_values=(('external_ids', {'iface-id': 'e1586e63-ee44-4717-ad47-886b1c3f01d0', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:eb:02:66', 'vm-uuid': '12351dbd-0f2e-40a6-a10a-f40f5b6250ef'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:46:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:46:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:20 user nova-compute[72081]: INFO os_vif [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:eb:02:66,bridge_name='br-int',has_traffic_filtering=True,id=e1586e63-ee44-4717-ad47-886b1c3f01d0,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape1586e63-ee')
Apr 12 03:46:20 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:46:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:46:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] No VIF found with MAC fa:16:3e:eb:02:66, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:46:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:21 user nova-compute[72081]: DEBUG nova.compute.manager [req-db957f91-23c4-4066-840b-5de9f7c3cfc3 req-b886d851-8af0-4577-9153-ab1121c74558 service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Received event network-vif-plugged-e1586e63-ee44-4717-ad47-886b1c3f01d0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:46:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-db957f91-23c4-4066-840b-5de9f7c3cfc3 req-b886d851-8af0-4577-9153-ab1121c74558 service nova] Acquiring lock "12351dbd-0f2e-40a6-a10a-f40f5b6250ef-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:46:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-db957f91-23c4-4066-840b-5de9f7c3cfc3 req-b886d851-8af0-4577-9153-ab1121c74558 service nova] Lock "12351dbd-0f2e-40a6-a10a-f40f5b6250ef-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:46:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-db957f91-23c4-4066-840b-5de9f7c3cfc3 req-b886d851-8af0-4577-9153-ab1121c74558 service nova] Lock "12351dbd-0f2e-40a6-a10a-f40f5b6250ef-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:46:21 user nova-compute[72081]: DEBUG nova.compute.manager [req-db957f91-23c4-4066-840b-5de9f7c3cfc3 req-b886d851-8af0-4577-9153-ab1121c74558 service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] No waiting events found dispatching network-vif-plugged-e1586e63-ee44-4717-ad47-886b1c3f01d0 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:46:21 user nova-compute[72081]: WARNING nova.compute.manager [req-db957f91-23c4-4066-840b-5de9f7c3cfc3 req-b886d851-8af0-4577-9153-ab1121c74558 service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Received unexpected event network-vif-plugged-e1586e63-ee44-4717-ad47-886b1c3f01d0 for instance with vm_state building and task_state spawning.
Apr 12 03:46:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:23 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271183.6724231, 12351dbd-0f2e-40a6-a10a-f40f5b6250ef => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:46:23 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] VM Resumed (Lifecycle Event)
Apr 12 03:46:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:46:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:46:23 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Instance spawned successfully.
Apr 12 03:46:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:46:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:46:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:46:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:46:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:46:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:46:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:46:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:46:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:46:23 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:46:23 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271183.6745813, 12351dbd-0f2e-40a6-a10a-f40f5b6250ef => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:46:23 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] VM Started (Lifecycle Event)
Apr 12 03:46:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:46:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:46:23 user nova-compute[72081]: DEBUG nova.compute.manager [req-05412171-2b7b-4db9-9206-68f6a060c0ba req-f2ec5664-769c-4813-b7bf-8b736dae000a service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Received event network-vif-plugged-e1586e63-ee44-4717-ad47-886b1c3f01d0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:46:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-05412171-2b7b-4db9-9206-68f6a060c0ba req-f2ec5664-769c-4813-b7bf-8b736dae000a service nova] Acquiring lock "12351dbd-0f2e-40a6-a10a-f40f5b6250ef-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:46:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-05412171-2b7b-4db9-9206-68f6a060c0ba req-f2ec5664-769c-4813-b7bf-8b736dae000a service nova] Lock "12351dbd-0f2e-40a6-a10a-f40f5b6250ef-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:46:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-05412171-2b7b-4db9-9206-68f6a060c0ba req-f2ec5664-769c-4813-b7bf-8b736dae000a service nova] Lock "12351dbd-0f2e-40a6-a10a-f40f5b6250ef-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:46:23 user nova-compute[72081]: DEBUG nova.compute.manager [req-05412171-2b7b-4db9-9206-68f6a060c0ba req-f2ec5664-769c-4813-b7bf-8b736dae000a service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] No waiting events found dispatching network-vif-plugged-e1586e63-ee44-4717-ad47-886b1c3f01d0 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:46:23 user nova-compute[72081]: WARNING nova.compute.manager [req-05412171-2b7b-4db9-9206-68f6a060c0ba req-f2ec5664-769c-4813-b7bf-8b736dae000a service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Received unexpected event network-vif-plugged-e1586e63-ee44-4717-ad47-886b1c3f01d0 for instance with vm_state building and task_state spawning.
Apr 12 03:46:23 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:46:23 user nova-compute[72081]: INFO nova.compute.manager [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Took 5.07 seconds to spawn the instance on the hypervisor.
Apr 12 03:46:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:46:23 user nova-compute[72081]: INFO nova.compute.manager [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Took 10.71 seconds to build instance.
Apr 12 03:46:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ed85b1cb-2d6f-4c13-8b1c-af31e76add19 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "12351dbd-0f2e-40a6-a10a-f40f5b6250ef" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 10.837s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:46:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:46:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:45 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:46 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:46:46 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:46:46 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:46:46 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:46:46 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:46:46 user nova-compute[72081]: INFO nova.compute.manager [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Terminating instance
Apr 12 03:46:46 user nova-compute[72081]: DEBUG nova.compute.manager [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:46:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG nova.compute.manager [req-eeb53517-9441-4ad3-9fa3-02c0939dd433 req-770f19ef-833a-4b27-8f60-b42177c64a7e service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received event network-vif-unplugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-eeb53517-9441-4ad3-9fa3-02c0939dd433 req-770f19ef-833a-4b27-8f60-b42177c64a7e service nova] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-eeb53517-9441-4ad3-9fa3-02c0939dd433 req-770f19ef-833a-4b27-8f60-b42177c64a7e service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-eeb53517-9441-4ad3-9fa3-02c0939dd433 req-770f19ef-833a-4b27-8f60-b42177c64a7e service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG nova.compute.manager [req-eeb53517-9441-4ad3-9fa3-02c0939dd433 req-770f19ef-833a-4b27-8f60-b42177c64a7e service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] No waiting events found dispatching network-vif-unplugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG nova.compute.manager [req-eeb53517-9441-4ad3-9fa3-02c0939dd433 req-770f19ef-833a-4b27-8f60-b42177c64a7e service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received event network-vif-unplugged-18f9f68c-eadd-4582-8f16-b88a13496852 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:47 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Instance destroyed successfully.
Apr 12 03:46:47 user nova-compute[72081]: DEBUG nova.objects.instance [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'resources' on Instance uuid 48977eb2-ce43-4577-a1cb-340e35703dbd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-12T03:33:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-2050314981',display_name='tempest-ServerActionsTestJSON-server-1847743592',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-2050314981',id=14,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKmrI9wy1CUIwhlKuXvzk0mD7Pt5nZFvI1CsqS0do6Lk76JQOvGl5lvx9rkz7UAM9WGocGXyWSOUOuKTtYZRD2HqKUbTN4VPOoliE1yuu+s0lx7vuZu45Mj7wDb2S8Ksyw==',key_name='tempest-keypair-364276420',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:44:09Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={rebuild='server'},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='815b65aa5ad9442db0a087c646f96bbd',ramdisk_id='',reservation_id='r-usz98205',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestJSON-1898059859',owner_user_name='tempest-ServerActionsTestJSON-1898059859-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:44:10Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a7f422f1a05a4a92ba387e424850b360',uuid=48977eb2-ce43-4577-a1cb-340e35703dbd,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converting VIF {"id": "18f9f68c-eadd-4582-8f16-b88a13496852", "address": "fa:16:3e:b3:c5:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18f9f68c-ea", "ovs_interfaceid": "18f9f68c-eadd-4582-8f16-b88a13496852", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG os_vif [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap18f9f68c-ea, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:46:47 user nova-compute[72081]: INFO os_vif [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:b3:c5:ca,bridge_name='br-int',has_traffic_filtering=True,id=18f9f68c-eadd-4582-8f16-b88a13496852,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18f9f68c-ea')
Apr 12 03:46:47 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YDhqrwbV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdg': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:46:47 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:46:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[cc5ee00b-36d6-4a82-9cee-c58b2076ba99]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:46:47 user nova-compute[72081]: No targets found.
Apr 12 03:46:47 user nova-compute[72081]: iSNS:
Apr 12 03:46:47 user nova-compute[72081]: No targets found.
Apr 12 03:46:47 user nova-compute[72081]: STATIC:
Apr 12 03:46:47 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27
Apr 12 03:46:47 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:46:47 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:46:47 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:46:47 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:46:47 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:46:47 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e
Apr 12 03:46:47 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:46:47 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:46:47 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068
Apr 12 03:46:47 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:46:47 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:46:47 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742
Apr 12 03:46:47 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:46:47 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:46:47 user nova-compute[72081]: FIRMWARE:
Apr 12 03:46:47 user nova-compute[72081]: No targets found.
Apr 12 03:46:47 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:46:47 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:46:47 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4b01aa8b-c8db-4311-a720-9d51ee5dcffc]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[95aac916-cacf-46a5-b225-69eb5a473acb]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:46:47 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:46:47 user nova-compute[72081]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:46:47 user nova-compute[72081]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:46:47 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:46:47 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:46:47 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:46:47 user nova-compute[72081]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 (non-flash)
Apr 12 03:46:47 user nova-compute[72081]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:46:47 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:46:47 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fac2825f0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742'): ({'sdg'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Removing single pathed devices sdg {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f8f51d97-087f-4cf3-b383-aa1834bedbb1]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdg {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdg" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb526098-9bc2-482a-ab31-9e254f85eb8b]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Flushing IO for device /dev/sdg {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdg {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdg" returned: 0 in 0.013s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[01fa3600-bf5b-42d3-b435-3263cd1a5cb2]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Remove SCSI device /dev/sdg with /sys/block/sdg/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdg/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdg/device/delete" returned: 0 in 0.040s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5c218388-5562-46b3-ac6c-4a31cddb4bf0]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Checking to see if SCSI volumes sdg have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] SCSI volumes sdg have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[13562f5b-f0ac-41ae-a4c1-0b0424d82cf1]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --logout" returned: 0 in 0.054s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[15c368a8-8f35-408c-83b5-a8d29168db63]: (4, ('Logging out of session [sid: 34, target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742, portal: 172.16.0.220,3260]\nLogout of [sid: 34, target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 34, target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742, portal: 172.16.0.220,3260]
Apr 12 03:46:47 user nova-compute[72081]: Logout of [sid: 34, target: iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742, portal: 172.16.0.220,3260] successful.
Apr 12 03:46:47 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8278d82d-cc19-4660-b4f1-b13d92e7d742 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1090daea-98a1-428c-b94a-afc609f18e52]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.304s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] <== disconnect_volume: return (306ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:46:47 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Deleting instance files /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd_del
Apr 12 03:46:47 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Deletion of /opt/stack/data/nova/instances/48977eb2-ce43-4577-a1cb-340e35703dbd_del complete
Apr 12 03:46:47 user nova-compute[72081]: INFO nova.compute.manager [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Took 1.16 seconds to destroy the instance on the hypervisor.
Apr 12 03:46:47 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:46:47 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:46:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:48 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:46:48 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Took 0.88 seconds to deallocate network for instance.
Apr 12 03:46:48 user nova-compute[72081]: DEBUG nova.compute.manager [req-85422d02-1cdf-482b-b50d-57646e11457f req-3e36be5f-dc45-44ab-b4e0-8f403677c523 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received event network-vif-deleted-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:46:49 user nova-compute[72081]: DEBUG nova.compute.manager [req-bbf4e3a8-2cef-4899-9fce-f74cbf9645d1 req-78423c9b-c22d-4eee-9734-22ecd1cc7c33 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:46:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bbf4e3a8-2cef-4899-9fce-f74cbf9645d1 req-78423c9b-c22d-4eee-9734-22ecd1cc7c33 service nova] Acquiring lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:46:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bbf4e3a8-2cef-4899-9fce-f74cbf9645d1 req-78423c9b-c22d-4eee-9734-22ecd1cc7c33 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:46:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bbf4e3a8-2cef-4899-9fce-f74cbf9645d1 req-78423c9b-c22d-4eee-9734-22ecd1cc7c33 service nova] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:46:49 user nova-compute[72081]: DEBUG nova.compute.manager [req-bbf4e3a8-2cef-4899-9fce-f74cbf9645d1 req-78423c9b-c22d-4eee-9734-22ecd1cc7c33 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] No waiting events found dispatching network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:46:49 user nova-compute[72081]: WARNING nova.compute.manager [req-bbf4e3a8-2cef-4899-9fce-f74cbf9645d1 req-78423c9b-c22d-4eee-9734-22ecd1cc7c33 service nova] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Received unexpected event network-vif-plugged-18f9f68c-eadd-4582-8f16-b88a13496852 for instance with vm_state active and task_state deleting.
Apr 12 03:46:49 user nova-compute[72081]: INFO nova.compute.manager [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Took 0.78 seconds to detach 1 volumes for instance.
Apr 12 03:46:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:46:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:46:49 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:46:49 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:46:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.277s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:46:49 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Deleted allocations for instance 48977eb2-ce43-4577-a1cb-340e35703dbd
Apr 12 03:46:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-10312cdd-3c4f-4aa1-9ada-eecab11a4063 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "48977eb2-ce43-4577-a1cb-340e35703dbd" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.273s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:46:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-3bef051e-2a65-4828-9e81-7f37bf52cabf req-06ca1f7d-add7-48e1-870e-12acc03c03f4 service nova] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Received event network-changed-e294fdef-5e1a-47fd-bcb1-08b22b4941b0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:46:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-3bef051e-2a65-4828-9e81-7f37bf52cabf req-06ca1f7d-add7-48e1-870e-12acc03c03f4 service nova] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Refreshing instance network info cache due to event network-changed-e294fdef-5e1a-47fd-bcb1-08b22b4941b0. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:46:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3bef051e-2a65-4828-9e81-7f37bf52cabf req-06ca1f7d-add7-48e1-870e-12acc03c03f4 service nova] Acquiring lock "refresh_cache-a264f3f7-b0ea-482a-a438-909f06a80726" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:46:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3bef051e-2a65-4828-9e81-7f37bf52cabf req-06ca1f7d-add7-48e1-870e-12acc03c03f4 service nova] Acquired lock "refresh_cache-a264f3f7-b0ea-482a-a438-909f06a80726" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:46:58 user nova-compute[72081]: DEBUG nova.network.neutron [req-3bef051e-2a65-4828-9e81-7f37bf52cabf req-06ca1f7d-add7-48e1-870e-12acc03c03f4 service nova] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Refreshing network info cache for port e294fdef-5e1a-47fd-bcb1-08b22b4941b0 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:46:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:46:58 user nova-compute[72081]: DEBUG nova.network.neutron [req-3bef051e-2a65-4828-9e81-7f37bf52cabf req-06ca1f7d-add7-48e1-870e-12acc03c03f4 service nova] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Updated VIF entry in instance network info cache for port e294fdef-5e1a-47fd-bcb1-08b22b4941b0. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:46:58 user nova-compute[72081]: DEBUG nova.network.neutron [req-3bef051e-2a65-4828-9e81-7f37bf52cabf req-06ca1f7d-add7-48e1-870e-12acc03c03f4 service nova] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Updating instance_info_cache with network_info: [{"id": "e294fdef-5e1a-47fd-bcb1-08b22b4941b0", "address": "fa:16:3e:4a:45:5f", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.163", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape294fdef-5e", "ovs_interfaceid": "e294fdef-5e1a-47fd-bcb1-08b22b4941b0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:46:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3bef051e-2a65-4828-9e81-7f37bf52cabf req-06ca1f7d-add7-48e1-870e-12acc03c03f4 service nova] Releasing lock "refresh_cache-a264f3f7-b0ea-482a-a438-909f06a80726" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "a264f3f7-b0ea-482a-a438-909f06a80726" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "a264f3f7-b0ea-482a-a438-909f06a80726" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG nova.objects.instance [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lazy-loading 'flavor' on Instance uuid a264f3f7-b0ea-482a-a438-909f06a80726 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "a264f3f7-b0ea-482a-a438-909f06a80726" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.052s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "a264f3f7-b0ea-482a-a438-909f06a80726" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "a264f3f7-b0ea-482a-a438-909f06a80726" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:46:59 user nova-compute[72081]: INFO nova.compute.manager [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Attaching volume 27f03268-3b13-4460-a92b-9de6d6f63d8e to /dev/vdb
Apr 12 03:46:59 user nova-compute[72081]: DEBUG os_brick.utils [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ce51ca1c-b73f-4a83-a41c-6f4e27799c3b]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2236fd47-33c9-4920-a741-5f6a5ffdf709]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8905e5d0-e713-4e82-a4ac-9fb4ec98e507]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[69045551-200d-4ed1-8e5d-50e1d4d1c254]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:46:59 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:46:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6a543662-1dae-4ffd-a69f-23729889a9b9]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG os_brick.utils [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] <== get_connector_properties: return (116ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:46:59 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Updating existing volume attachment record: f6cfa185-c561-4ada-90f5-61c2c663824c {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:47:02 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681271207.488114, 48977eb2-ce43-4577-a1cb-340e35703dbd => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:47:02 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] VM Stopped (Lifecycle Event)
Apr 12 03:47:02 user nova-compute[72081]: DEBUG nova.compute.manager [None req-d02980b5-3b9d-45c7-b761-74470d21df42 None None] [instance: 48977eb2-ce43-4577-a1cb-340e35703dbd] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:47:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'PSymFvTd', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:47:04 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:47:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:47:04 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:47:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:47:04 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[cc8412f9-ecaf-4c23-ac18-aef84a8f5770]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:47:04 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[81e35f64-821c-43d4-8f67-e76b2c8b6e04]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e] added
Apr 12 03:47:04 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[404c05b3-3b16-45c3-8a9a-379bc535fc4f]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[882ecc24-3df0-4b3b-a327-642f3b05db3d]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v PSymFvTd {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v PSymFvTd" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b1e99b89-0d34-4524-95b6-3dbc8b09431f]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'PSymFvTd'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e45bafc6-d025-4164-b135-bc6eb382d34f]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[21e07f9d-fc4e-4bbc-a329-ed8718483672]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:47:04 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:47:04 user nova-compute[72081]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:47:04 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:47:04 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:47:04 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:47:04 user nova-compute[72081]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:47:04 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:47:04 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e -p 172.16.0.220:3260 --login" returned: 0 in 0.064s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c1d1cdc4-840d-4ee5-946c-9dba903cab23]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e, portal: 172.16.0.220,3260]
Apr 12 03:47:04 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e, portal: 172.16.0.220,3260] successful.
Apr 12 03:47:04 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[98d79e35-547e-482c-ad91-ef0844d164ef]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[530a3ec5-d11e-4728-8ff6-6903d5df92ce]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:47:04 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:47:04 user nova-compute[72081]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:47:04 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:47:04 user nova-compute[72081]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e (non-flash)
Apr 12 03:47:04 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:47:04 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:47:04 user nova-compute[72081]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:47:04 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:47:04 user nova-compute[72081]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e (non-flash)
Apr 12 03:47:04 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.283s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] HCTL ('37', '-', '-', 0) found on session 38 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Scanning host 37 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[17c73610-aaad-48dc-bfa0-646b2c5aef14]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Searching for a device in session 38 and hctl ['37', '*', '*', 0] yield: sdf {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:47:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Connected to sdf using {'target_iqn': 'iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'PSymFvTd', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:47:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.316s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:47:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] <== connect_volume: return (1318ms) {'type': 'block', 'scsi_wwn': '26334626337373964', 'path': '/dev/sdf'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:47:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26334626337373964', 'path': '/dev/sdf'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:47:05 user nova-compute[72081]: DEBUG nova.objects.instance [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lazy-loading 'flavor' on Instance uuid a264f3f7-b0ea-482a-a438-909f06a80726 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:47:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] attach device xml: <disk type="block" device="disk">
Apr 12 03:47:05 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:47:05 user nova-compute[72081]:   <source dev="/dev/sdf"/>
Apr 12 03:47:05 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:47:05 user nova-compute[72081]:   <serial>27f03268-3b13-4460-a92b-9de6d6f63d8e</serial>
Apr 12 03:47:05 user nova-compute[72081]: </disk>
Apr 12 03:47:05 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 03:47:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:47:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:47:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] No VIF found with MAC fa:16:3e:4a:45:5f, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:47:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-be037fe2-7b9e-45c1-aa23-c6d28c4cbed7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "a264f3f7-b0ea-482a-a438-909f06a80726" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.308s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "a264f3f7-b0ea-482a-a438-909f06a80726" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "a264f3f7-b0ea-482a-a438-909f06a80726" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:06 user nova-compute[72081]: INFO nova.compute.manager [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Detaching volume 27f03268-3b13-4460-a92b-9de6d6f63d8e
Apr 12 03:47:06 user nova-compute[72081]: INFO nova.virt.block_device [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Attempting to driver detach volume 27f03268-3b13-4460-a92b-9de6d6f63d8e from mountpoint /dev/vdb
Apr 12 03:47:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Attempting to detach device vdb from instance a264f3f7-b0ea-482a-a438-909f06a80726 from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 03:47:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:47:06 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:47:06 user nova-compute[72081]:   <source dev="/dev/sdf"/>
Apr 12 03:47:06 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:47:06 user nova-compute[72081]:   <serial>27f03268-3b13-4460-a92b-9de6d6f63d8e</serial>
Apr 12 03:47:06 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 03:47:06 user nova-compute[72081]: </disk>
Apr 12 03:47:06 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:47:06 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Successfully detached device vdb from instance a264f3f7-b0ea-482a-a438-909f06a80726 from the persistent domain config.
Apr 12 03:47:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance a264f3f7-b0ea-482a-a438-909f06a80726 from the live domain config. {{(pid=72081) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 12 03:47:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:47:06 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:47:06 user nova-compute[72081]:   <source dev="/dev/sdf"/>
Apr 12 03:47:06 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:47:06 user nova-compute[72081]:   <serial>27f03268-3b13-4460-a92b-9de6d6f63d8e</serial>
Apr 12 03:47:06 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 03:47:06 user nova-compute[72081]: </disk>
Apr 12 03:47:06 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Received event <DeviceRemovedEvent: 1681271227.1075666, a264f3f7-b0ea-482a-a438-909f06a80726 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72081) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance a264f3f7-b0ea-482a-a438-909f06a80726 {{(pid=72081) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 12 03:47:07 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Successfully detached device vdb from instance a264f3f7-b0ea-482a-a438-909f06a80726 from the live domain config.
Apr 12 03:47:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'PSymFvTd', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:47:07 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:47:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8044f787-d783-48d9-b0e7-a59bdf418eff]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:47:07 user nova-compute[72081]: No targets found.
Apr 12 03:47:07 user nova-compute[72081]: iSNS:
Apr 12 03:47:07 user nova-compute[72081]: No targets found.
Apr 12 03:47:07 user nova-compute[72081]: STATIC:
Apr 12 03:47:07 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e
Apr 12 03:47:07 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:47:07 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:47:07 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27
Apr 12 03:47:07 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:47:07 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:47:07 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:47:07 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:47:07 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:47:07 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e
Apr 12 03:47:07 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:47:07 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:47:07 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068
Apr 12 03:47:07 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:47:07 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:47:07 user nova-compute[72081]: FIRMWARE:
Apr 12 03:47:07 user nova-compute[72081]: No targets found.
Apr 12 03:47:07 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:47:07 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:47:07 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0e119e71-73bd-42a0-b96b-73807925811e]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f9249b92-79b8-4509-adde-2c13536f5c69]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:47:07 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:47:07 user nova-compute[72081]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:47:07 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:47:07 user nova-compute[72081]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e (non-flash)
Apr 12 03:47:07 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:47:07 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:47:07 user nova-compute[72081]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:47:07 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:47:07 user nova-compute[72081]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e (non-flash)
Apr 12 03:47:07 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3f9412c280>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e'): ({'sdf'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Removing single pathed devices sdf {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[af239799-6d9e-42dc-9bfd-62430e4cbf3f]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdf {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdf" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c652e4e-3025-4cab-879b-75059fee17d9]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Flushing IO for device /dev/sdf {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdf {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdf" returned: 0 in 0.013s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[675cdf49-712c-4d13-9301-3eec54dd3bf4]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Remove SCSI device /dev/sdf with /sys/block/sdf/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdf/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdf/device/delete" returned: 0 in 0.042s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[976694b4-b7c1-4e1d-81ed-8b07b3102d38]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Checking to see if SCSI volumes sdf have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] SCSI volumes sdf have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[528c8224-1a4b-41cb-a1d0-71757359ff1d]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e -p 172.16.0.220:3260 --logout" returned: 0 in 0.065s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2464ba56-2714-4036-8271-39dccb040a02]: (4, ('Logging out of session [sid: 38, target: iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e, portal: 172.16.0.220,3260]\nLogout of [sid: 38, target: iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 38, target: iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e, portal: 172.16.0.220,3260]
Apr 12 03:47:07 user nova-compute[72081]: Logout of [sid: 38, target: iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e, portal: 172.16.0.220,3260] successful.
Apr 12 03:47:07 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:27f03268-3b13-4460-a92b-9de6d6f63d8e -p 172.16.0.220:3260 --op delete" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7d37f0f9-1dd9-4af4-91e7-0d2c42529c0d]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.326s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] <== disconnect_volume: return (327ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:47:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:08 user nova-compute[72081]: DEBUG nova.objects.instance [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lazy-loading 'flavor' on Instance uuid a264f3f7-b0ea-482a-a438-909f06a80726 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:47:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2d658e96-c6d5-4daa-b094-674415537bfd tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "a264f3f7-b0ea-482a-a438-909f06a80726" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.620s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:08 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:47:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:12 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:47:12 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:47:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:13 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "614495f2-f108-4759-be60-bc6cc689c5b7" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "614495f2-f108-4759-be60-bc6cc689c5b7" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:14 user nova-compute[72081]: INFO nova.compute.manager [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Terminating instance
Apr 12 03:47:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG nova.compute.manager [req-07aff617-75f1-42d7-b08b-4fdcfc16b532 req-693f2e13-2bb5-4c3f-a786-4f0e038272c5 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received event network-vif-unplugged-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-07aff617-75f1-42d7-b08b-4fdcfc16b532 req-693f2e13-2bb5-4c3f-a786-4f0e038272c5 service nova] Acquiring lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-07aff617-75f1-42d7-b08b-4fdcfc16b532 req-693f2e13-2bb5-4c3f-a786-4f0e038272c5 service nova] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-07aff617-75f1-42d7-b08b-4fdcfc16b532 req-693f2e13-2bb5-4c3f-a786-4f0e038272c5 service nova] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG nova.compute.manager [req-07aff617-75f1-42d7-b08b-4fdcfc16b532 req-693f2e13-2bb5-4c3f-a786-4f0e038272c5 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] No waiting events found dispatching network-vif-unplugged-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG nova.compute.manager [req-07aff617-75f1-42d7-b08b-4fdcfc16b532 req-693f2e13-2bb5-4c3f-a786-4f0e038272c5 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received event network-vif-unplugged-b9acaee6-a684-4827-bc91-1c6219782efe for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:14 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Instance destroyed successfully.
Apr 12 03:47:14 user nova-compute[72081]: DEBUG nova.objects.instance [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lazy-loading 'resources' on Instance uuid 614495f2-f108-4759-be60-bc6cc689c5b7 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:41:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1156151605',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1156151605',id=30,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:43:40Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='6e1fa352a8c144a1aada7ce0d8818429',ramdisk_id='',reservation_id='r-r2n25ts0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-590302458',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:45:29Z,user_data=None,user_id='d99be569057f432484e4a60b46882bbd',uuid=614495f2-f108-4759-be60-bc6cc689c5b7,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "b9acaee6-a684-4827-bc91-1c6219782efe", "address": "fa:16:3e:d4:dc:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb9acaee6-a6", "ovs_interfaceid": "b9acaee6-a684-4827-bc91-1c6219782efe", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converting VIF {"id": "b9acaee6-a684-4827-bc91-1c6219782efe", "address": "fa:16:3e:d4:dc:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb9acaee6-a6", "ovs_interfaceid": "b9acaee6-a684-4827-bc91-1c6219782efe", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:d4:dc:db,bridge_name='br-int',has_traffic_filtering=True,id=b9acaee6-a684-4827-bc91-1c6219782efe,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb9acaee6-a6') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG os_vif [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:d4:dc:db,bridge_name='br-int',has_traffic_filtering=True,id=b9acaee6-a684-4827-bc91-1c6219782efe,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb9acaee6-a6') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapb9acaee6-a6, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:47:14 user nova-compute[72081]: INFO os_vif [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:d4:dc:db,bridge_name='br-int',has_traffic_filtering=True,id=b9acaee6-a684-4827-bc91-1c6219782efe,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb9acaee6-a6')
Apr 12 03:47:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'fbzrSZhe', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:47:14 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:47:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_power_states {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bbde27ee-32c7-49a2-8c91-ac8835fe8ba6]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:47:14 user nova-compute[72081]: No targets found.
Apr 12 03:47:14 user nova-compute[72081]: iSNS:
Apr 12 03:47:14 user nova-compute[72081]: No targets found.
Apr 12 03:47:14 user nova-compute[72081]: STATIC:
Apr 12 03:47:14 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27
Apr 12 03:47:14 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:47:14 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:47:14 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:47:14 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:47:14 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:47:14 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e
Apr 12 03:47:14 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:47:14 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:47:14 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068
Apr 12 03:47:14 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:47:14 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:47:14 user nova-compute[72081]: FIRMWARE:
Apr 12 03:47:14 user nova-compute[72081]: No targets found.
Apr 12 03:47:14 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:47:14 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:47:14 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4774240d-16e9-49b5-8731-174e90741b94]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Triggering sync for uuid 2d0904c3-74e0-47bb-a832-758a7035b2f5 {{(pid=72081) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Triggering sync for uuid bcd267d2-5b2c-4ed2-b634-0210fae94dfd {{(pid=72081) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Triggering sync for uuid 3d4407f4-e078-4777-a110-cb3f922be2e8 {{(pid=72081) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Triggering sync for uuid 614495f2-f108-4759-be60-bc6cc689c5b7 {{(pid=72081) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Triggering sync for uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Triggering sync for uuid a264f3f7-b0ea-482a-a438-909f06a80726 {{(pid=72081) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Triggering sync for uuid 7c2344ea-3ed4-40f5-b318-6fa8588b1fed {{(pid=72081) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Triggering sync for uuid 12351dbd-0f2e-40a6-a10a-f40f5b6250ef {{(pid=72081) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "2d0904c3-74e0-47bb-a832-758a7035b2f5" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "2d0904c3-74e0-47bb-a832-758a7035b2f5" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "3d4407f4-e078-4777-a110-cb3f922be2e8" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "3d4407f4-e078-4777-a110-cb3f922be2e8" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "614495f2-f108-4759-be60-bc6cc689c5b7" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "a264f3f7-b0ea-482a-a438-909f06a80726" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "a264f3f7-b0ea-482a-a438-909f06a80726" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "12351dbd-0f2e-40a6-a10a-f40f5b6250ef" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "12351dbd-0f2e-40a6-a10a-f40f5b6250ef" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f05d8e8f-887d-4662-a6ba-3ab1172961a5]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:47:14 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:47:14 user nova-compute[72081]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:47:14 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:47:14 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:47:14 user nova-compute[72081]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e (non-flash)
Apr 12 03:47:14 user nova-compute[72081]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:47:14 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:47:14 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3f7441b880>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e'): ({'sdc'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Removing single pathed devices sdc {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:47:14 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d006cca4-360b-4288-bca6-b41a1335f11f]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "2d0904c3-74e0-47bb-a832-758a7035b2f5" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.047s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.047s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "3d4407f4-e078-4777-a110-cb3f922be2e8" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.051s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.052s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "a264f3f7-b0ea-482a-a438-909f06a80726" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.057s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[895c15d9-1671-4045-8e3f-d980f9027b4f]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Flushing IO for device /dev/sdc {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.066s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "12351dbd-0f2e-40a6-a10a-f40f5b6250ef" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.068s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb49df07-7171-48fb-9774-544c60bfa681]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.054s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7f1f9550-1c8b-46f2-bf2b-7b70240e4504]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] SCSI volumes sdc have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-SSCST_BIO_Storage_7028adfd', '/dev/disk/by-id/scsi-1SCST_BIO7028adfd-07534774499cd413', '/dev/disk/by-id/scsi-23037353334373734') {{(pid=82083) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Apr 12 03:47:15 user nova-compute[72081]: WARNING os_brick.exception [-] Unlink failed for /dev/disk/by-id/scsi-SSCST_BIO_Storage_7028adfd
Apr 12 03:47:15 user nova-compute[72081]: WARNING os_brick.exception [-] Unlink failed for /dev/disk/by-id/scsi-1SCST_BIO7028adfd-07534774499cd413
Apr 12 03:47:15 user nova-compute[72081]: WARNING os_brick.exception [-] Unlink failed for /dev/disk/by-id/scsi-23037353334373734
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ef091d4b-ea03-48cd-a7b9-996bb64383d7]: (4, None) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[05c5baed-4437-438d-8dfb-4b6164c0ceae]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --logout" returned: 0 in 0.073s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[839ee745-59cd-406e-9a59-d9e5c14337fd]: (4, ('Logging out of session [sid: 28, target: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e, portal: 172.16.0.220,3260]\nLogout of [sid: 28, target: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 28, target: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e, portal: 172.16.0.220,3260]
Apr 12 03:47:15 user nova-compute[72081]: Logout of [sid: 28, target: iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e, portal: 172.16.0.220,3260] successful.
Apr 12 03:47:15 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:54a9604a-4853-414c-850e-51745d6fcd0e -p 172.16.0.220:3260 --op delete" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[38c6e847-de03-41da-b96f-5997171691e0]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.345s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] <== disconnect_volume: return (346ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:47:15 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Deleting instance files /opt/stack/data/nova/instances/614495f2-f108-4759-be60-bc6cc689c5b7_del
Apr 12 03:47:15 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Deletion of /opt/stack/data/nova/instances/614495f2-f108-4759-be60-bc6cc689c5b7_del complete
Apr 12 03:47:15 user nova-compute[72081]: INFO nova.compute.manager [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Took 1.19 seconds to destroy the instance on the hypervisor.
Apr 12 03:47:15 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:47:15 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:47:15 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Took 0.49 seconds to deallocate network for instance.
Apr 12 03:47:15 user nova-compute[72081]: DEBUG nova.compute.manager [req-c04e8c98-9c50-4ba3-94b1-f07f9956c958 req-737776f2-353d-425f-bf0e-df8cdd7c41db service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received event network-vif-deleted-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:47:16 user nova-compute[72081]: DEBUG nova.compute.manager [req-993d2bc9-1e51-4fdf-9531-d9b04805af8a req-3a91773a-0cb4-404a-b289-63676ccbed19 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received event network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:47:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-993d2bc9-1e51-4fdf-9531-d9b04805af8a req-3a91773a-0cb4-404a-b289-63676ccbed19 service nova] Acquiring lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-993d2bc9-1e51-4fdf-9531-d9b04805af8a req-3a91773a-0cb4-404a-b289-63676ccbed19 service nova] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-993d2bc9-1e51-4fdf-9531-d9b04805af8a req-3a91773a-0cb4-404a-b289-63676ccbed19 service nova] Lock "614495f2-f108-4759-be60-bc6cc689c5b7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:16 user nova-compute[72081]: DEBUG nova.compute.manager [req-993d2bc9-1e51-4fdf-9531-d9b04805af8a req-3a91773a-0cb4-404a-b289-63676ccbed19 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] No waiting events found dispatching network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:47:16 user nova-compute[72081]: WARNING nova.compute.manager [req-993d2bc9-1e51-4fdf-9531-d9b04805af8a req-3a91773a-0cb4-404a-b289-63676ccbed19 service nova] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Received unexpected event network-vif-plugged-b9acaee6-a684-4827-bc91-1c6219782efe for instance with vm_state active and task_state deleting.
Apr 12 03:47:16 user nova-compute[72081]: INFO nova.compute.manager [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Took 0.78 seconds to detach 1 volumes for instance.
Apr 12 03:47:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:16 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:47:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:16 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:47:16 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:47:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.275s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.251s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:16 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:47:16 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Deleted allocations for instance 614495f2-f108-4759-be60-bc6cc689c5b7
Apr 12 03:47:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2987552d-130d-4ca1-b89d-28e40d748ee3 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "614495f2-f108-4759-be60-bc6cc689c5b7" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.916s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "614495f2-f108-4759-be60-bc6cc689c5b7" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 2.062s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:17 user nova-compute[72081]: INFO nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] During sync_power_state the instance has a pending task (deleting). Skip.
Apr 12 03:47:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "614495f2-f108-4759-be60-bc6cc689c5b7" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json" returned: 0 in 0.164s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:47:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:47:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.129s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdd (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:47:18 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:47:18 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:47:18 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8107MB free_disk=25.931438446044922GB free_vcpus=5 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:47:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:19 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 2d0904c3-74e0-47bb-a832-758a7035b2f5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:47:19 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance bcd267d2-5b2c-4ed2-b634-0210fae94dfd actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:47:19 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 3d4407f4-e078-4777-a110-cb3f922be2e8 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:47:19 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 606425bb-8968-41f9-b3a5-80b6fd188a67 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:47:19 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance a264f3f7-b0ea-482a-a438-909f06a80726 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:47:19 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 7c2344ea-3ed4-40f5-b318-6fa8588b1fed actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:47:19 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 12351dbd-0f2e-40a6-a10a-f40f5b6250ef actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:47:19 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 7 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:47:19 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=1408MB phys_disk=40GB used_disk=5GB total_vcpus=12 used_vcpus=7 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:47:19 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:47:19 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:47:19 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:47:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.360s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:20 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:47:20 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:47:20 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:47:20 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:47:20 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:47:20 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:47:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-bcd267d2-5b2c-4ed2-b634-0210fae94dfd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:47:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-bcd267d2-5b2c-4ed2-b634-0210fae94dfd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:47:20 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 03:47:21 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Updating instance_info_cache with network_info: [{"id": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "address": "fa:16:3e:9d:0c:c2", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa4385731-05", "ovs_interfaceid": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:47:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-bcd267d2-5b2c-4ed2-b634-0210fae94dfd" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:47:21 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG nova.compute.manager [req-3811ff7a-3d4b-4ead-909c-37cdfedc2a1d req-470931f3-35f2-4d1f-8197-02cacc82530c service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Received event network-changed-c2ac7de8-155e-4d88-8534-5a5291bd18dc {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG nova.compute.manager [req-3811ff7a-3d4b-4ead-909c-37cdfedc2a1d req-470931f3-35f2-4d1f-8197-02cacc82530c service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Refreshing instance network info cache due to event network-changed-c2ac7de8-155e-4d88-8534-5a5291bd18dc. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3811ff7a-3d4b-4ead-909c-37cdfedc2a1d req-470931f3-35f2-4d1f-8197-02cacc82530c service nova] Acquiring lock "refresh_cache-7c2344ea-3ed4-40f5-b318-6fa8588b1fed" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3811ff7a-3d4b-4ead-909c-37cdfedc2a1d req-470931f3-35f2-4d1f-8197-02cacc82530c service nova] Acquired lock "refresh_cache-7c2344ea-3ed4-40f5-b318-6fa8588b1fed" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG nova.network.neutron [req-3811ff7a-3d4b-4ead-909c-37cdfedc2a1d req-470931f3-35f2-4d1f-8197-02cacc82530c service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Refreshing network info cache for port c2ac7de8-155e-4d88-8534-5a5291bd18dc {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG nova.objects.instance [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lazy-loading 'flavor' on Instance uuid 7c2344ea-3ed4-40f5-b318-6fa8588b1fed {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.076s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:24 user nova-compute[72081]: INFO nova.compute.manager [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Attaching volume ce7f211d-d195-4fd9-936a-eb3ce8c33470 to /dev/vdb
Apr 12 03:47:24 user nova-compute[72081]: DEBUG os_brick.utils [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8a6d84e0-eeb6-46b4-8e21-48fd428f9e4e]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d9741a6b-2fb7-4bf0-b076-a544a7952486]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[fb6cbf86-91cc-49b4-9997-0e1d5a3abf69]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[766eb7ce-1a5c-4495-8956-c7a684b52172]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:47:24 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:47:24 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c335cf5a-b2a1-42a6-822b-322299bb68fb]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG os_brick.utils [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] <== get_connector_properties: return (145ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Updating existing volume attachment record: a611ade2-82a0-4cff-8799-b6ba2681822b {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG nova.network.neutron [req-3811ff7a-3d4b-4ead-909c-37cdfedc2a1d req-470931f3-35f2-4d1f-8197-02cacc82530c service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Updated VIF entry in instance network info cache for port c2ac7de8-155e-4d88-8534-5a5291bd18dc. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG nova.network.neutron [req-3811ff7a-3d4b-4ead-909c-37cdfedc2a1d req-470931f3-35f2-4d1f-8197-02cacc82530c service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Updating instance_info_cache with network_info: [{"id": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "address": "fa:16:3e:6d:2b:fe", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.98", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2ac7de8-15", "ovs_interfaceid": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3811ff7a-3d4b-4ead-909c-37cdfedc2a1d req-470931f3-35f2-4d1f-8197-02cacc82530c service nova] Releasing lock "refresh_cache-7c2344ea-3ed4-40f5-b318-6fa8588b1fed" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:47:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:47:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:28 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:47:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'KTZqvVls', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:47:28 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:47:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:47:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:47:28 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:47:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:47:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:47:28 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:47:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9550d76d-9a2b-4690-84f7-bf692515d2ee]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:47:28 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d6ef7625-0fd7-4100-9ff3-b64f53cef52f]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470] added
Apr 12 03:47:28 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a456adb3-df30-42e9-b21d-26d12ee258c8]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3a533c63-a8ef-453c-b61e-86c4487bef6e]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v KTZqvVls {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v KTZqvVls" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:28 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4be18a3a-8767-4dfe-b682-afe06e0373f1]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:28 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'KTZqvVls'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[adf54676-9293-4a86-84bc-70d835446695]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4ffacc19-3311-4990-b726-90582f42891f]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:47:29 user nova-compute[72081]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:47:29 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:47:29 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:47:29 user nova-compute[72081]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:47:29 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:47:29 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 -p 172.16.0.220:3260 --login" returned: 0 in 0.059s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f9113b08-5d3c-4a53-8dee-421572231dac]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470, portal: 172.16.0.220,3260]
Apr 12 03:47:29 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470, portal: 172.16.0.220,3260] successful.
Apr 12 03:47:29 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9f07c9e7-4be6-4758-ae19-261464704ddb]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4f42fd37-a00a-4c80-994f-e5a9a28f4dbb]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:47:29 user nova-compute[72081]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:47:29 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:47:29 user nova-compute[72081]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 (non-flash)
Apr 12 03:47:29 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:47:29 user nova-compute[72081]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:47:29 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:47:29 user nova-compute[72081]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 (non-flash)
Apr 12 03:47:29 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.275s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] HCTL ('34', '-', '-', 0) found on session 39 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[65579e7f-921c-41e5-9046-bd95cbc45191]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Searching for a device in session 39 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'KTZqvVls', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681271234.8538907, 614495f2-f108-4759-be60-bc6cc689c5b7 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:47:29 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] VM Stopped (Lifecycle Event)
Apr 12 03:47:29 user nova-compute[72081]: DEBUG nova.compute.manager [None req-5fa7a198-bac8-4805-b348-e63b0ee53bb6 None None] [instance: 614495f2-f108-4759-be60-bc6cc689c5b7] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:47:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.310s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:47:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] <== connect_volume: return (1312ms) {'type': 'block', 'scsi_wwn': '23136323265363636', 'path': '/dev/sdc'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:47:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23136323265363636', 'path': '/dev/sdc'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:47:30 user nova-compute[72081]: DEBUG nova.objects.instance [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lazy-loading 'flavor' on Instance uuid 7c2344ea-3ed4-40f5-b318-6fa8588b1fed {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:47:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] attach device xml: <disk type="block" device="disk">
Apr 12 03:47:30 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:47:30 user nova-compute[72081]:   <source dev="/dev/sdc"/>
Apr 12 03:47:30 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:47:30 user nova-compute[72081]:   <serial>ce7f211d-d195-4fd9-936a-eb3ce8c33470</serial>
Apr 12 03:47:30 user nova-compute[72081]: </disk>
Apr 12 03:47:30 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 03:47:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:47:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:47:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] No VIF found with MAC fa:16:3e:6d:2b:fe, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:47:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3ce42a08-5165-47af-94cc-74737dfc40bc tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.238s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:32 user nova-compute[72081]: INFO nova.compute.manager [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Detaching volume ce7f211d-d195-4fd9-936a-eb3ce8c33470
Apr 12 03:47:32 user nova-compute[72081]: INFO nova.virt.block_device [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Attempting to driver detach volume ce7f211d-d195-4fd9-936a-eb3ce8c33470 from mountpoint /dev/vdb
Apr 12 03:47:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Attempting to detach device vdb from instance 7c2344ea-3ed4-40f5-b318-6fa8588b1fed from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:47:32 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:47:32 user nova-compute[72081]:   <source dev="/dev/sdc"/>
Apr 12 03:47:32 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:47:32 user nova-compute[72081]:   <serial>ce7f211d-d195-4fd9-936a-eb3ce8c33470</serial>
Apr 12 03:47:32 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 03:47:32 user nova-compute[72081]: </disk>
Apr 12 03:47:32 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:47:32 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Successfully detached device vdb from instance 7c2344ea-3ed4-40f5-b318-6fa8588b1fed from the persistent domain config.
Apr 12 03:47:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 7c2344ea-3ed4-40f5-b318-6fa8588b1fed from the live domain config. {{(pid=72081) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:47:32 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:47:32 user nova-compute[72081]:   <source dev="/dev/sdc"/>
Apr 12 03:47:32 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:47:32 user nova-compute[72081]:   <serial>ce7f211d-d195-4fd9-936a-eb3ce8c33470</serial>
Apr 12 03:47:32 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 03:47:32 user nova-compute[72081]: </disk>
Apr 12 03:47:32 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Received event <DeviceRemovedEvent: 1681271252.8149328, 7c2344ea-3ed4-40f5-b318-6fa8588b1fed => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72081) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 7c2344ea-3ed4-40f5-b318-6fa8588b1fed {{(pid=72081) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 12 03:47:32 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Successfully detached device vdb from instance 7c2344ea-3ed4-40f5-b318-6fa8588b1fed from the live domain config.
Apr 12 03:47:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'KTZqvVls', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:47:32 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:47:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ca2f9d9d-2f88-4d7f-af94-a8036c3c47fe]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:47:32 user nova-compute[72081]: No targets found.
Apr 12 03:47:32 user nova-compute[72081]: iSNS:
Apr 12 03:47:32 user nova-compute[72081]: No targets found.
Apr 12 03:47:32 user nova-compute[72081]: STATIC:
Apr 12 03:47:32 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27
Apr 12 03:47:32 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:47:32 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:47:32 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:47:32 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:47:32 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:47:32 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068
Apr 12 03:47:32 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:47:32 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:47:32 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470
Apr 12 03:47:32 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:47:32 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:47:32 user nova-compute[72081]: FIRMWARE:
Apr 12 03:47:32 user nova-compute[72081]: No targets found.
Apr 12 03:47:32 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:47:32 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:47:32 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ef4ad5b4-db82-4ee7-87bd-282f31d02a36]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3826f3b5-b35a-476a-95e7-85f496a55e73]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:47:32 user nova-compute[72081]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:47:32 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:47:32 user nova-compute[72081]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 (non-flash)
Apr 12 03:47:32 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:47:32 user nova-compute[72081]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:47:32 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:47:32 user nova-compute[72081]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 (non-flash)
Apr 12 03:47:32 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3f9416c310>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470'): ({'sdc'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Removing single pathed devices sdc {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b0897b86-5f64-4cb7-82be-1732cf8cebec]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[78a32f22-a8af-4637-b0f5-12921a52433a]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Flushing IO for device /dev/sdc {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.012s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[23fe85aa-93eb-405c-901c-18857b36ec72]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:47:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.044s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:33 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3c7fb160-f30c-43ba-83ca-47ac2e81abce]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:33 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:47:33 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] SCSI volumes sdc have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:47:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:47:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:33 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b2fb9893-14b9-4ef1-b7ec-bb5019a74629]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 -p 172.16.0.220:3260 --logout" returned: 0 in 0.063s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:33 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[da69d971-10fc-4f84-a871-725abce59f6a]: (4, ('Logging out of session [sid: 39, target: iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470, portal: 172.16.0.220,3260]\nLogout of [sid: 39, target: iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 39, target: iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470, portal: 172.16.0.220,3260]
Apr 12 03:47:33 user nova-compute[72081]: Logout of [sid: 39, target: iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470, portal: 172.16.0.220,3260] successful.
Apr 12 03:47:33 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ce7f211d-d195-4fd9-936a-eb3ce8c33470 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:33 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[318bea49-d2ec-4fe7-8a96-9114451df433]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.317s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:47:33 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] <== disconnect_volume: return (318ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:47:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:47:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:33 user nova-compute[72081]: DEBUG nova.objects.instance [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lazy-loading 'flavor' on Instance uuid 7c2344ea-3ed4-40f5-b318-6fa8588b1fed {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:47:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c45cdc09-027b-49e4-a87b-804fb968313f tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.716s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "8d533f5f-dbed-4878-a9b1-2ce4fae8fa13" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "8d533f5f-dbed-4878-a9b1-2ce4fae8fa13" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:47:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:47:38 user nova-compute[72081]: INFO nova.compute.claims [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Claim successful on node user
Apr 12 03:47:38 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:47:38 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:47:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.350s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:47:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:47:38 user nova-compute[72081]: DEBUG nova.network.neutron [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:47:38 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:47:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:47:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:47:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:47:38 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Creating image(s)
Apr 12 03:47:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "/opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "/opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "/opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.040s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:39 user nova-compute[72081]: DEBUG nova.policy [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'a7f422f1a05a4a92ba387e424850b360', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '815b65aa5ad9442db0a087c646f96bbd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:47:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.133s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.132s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk 1073741824" returned: 0 in 0.046s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.185s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.129s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:39 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Checking if we can resize image /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:47:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:39 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Cannot resize image /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:47:39 user nova-compute[72081]: DEBUG nova.objects.instance [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'migration_context' on Instance uuid 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:47:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:47:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Ensure instance console log exists: /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:47:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:39 user nova-compute[72081]: DEBUG nova.network.neutron [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Successfully created port: 2ef93926-6cf3-4d57-be98-ab555ac58b4c {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:47:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.network.neutron [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Successfully updated port: 2ef93926-6cf3-4d57-be98-ab555ac58b4c {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "refresh_cache-8d533f5f-dbed-4878-a9b1-2ce4fae8fa13" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquired lock "refresh_cache-8d533f5f-dbed-4878-a9b1-2ce4fae8fa13" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.network.neutron [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.compute.manager [req-67892a7d-9e65-4e48-9832-3c57ae4ac7d1 req-c8bc6b48-7ae8-44c7-9aff-c7a18819c7ed service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Received event network-changed-2ef93926-6cf3-4d57-be98-ab555ac58b4c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.compute.manager [req-67892a7d-9e65-4e48-9832-3c57ae4ac7d1 req-c8bc6b48-7ae8-44c7-9aff-c7a18819c7ed service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Refreshing instance network info cache due to event network-changed-2ef93926-6cf3-4d57-be98-ab555ac58b4c. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-67892a7d-9e65-4e48-9832-3c57ae4ac7d1 req-c8bc6b48-7ae8-44c7-9aff-c7a18819c7ed service nova] Acquiring lock "refresh_cache-8d533f5f-dbed-4878-a9b1-2ce4fae8fa13" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.network.neutron [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.network.neutron [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Updating instance_info_cache with network_info: [{"id": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "address": "fa:16:3e:e2:fd:44", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2ef93926-6c", "ovs_interfaceid": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Releasing lock "refresh_cache-8d533f5f-dbed-4878-a9b1-2ce4fae8fa13" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Instance network_info: |[{"id": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "address": "fa:16:3e:e2:fd:44", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2ef93926-6c", "ovs_interfaceid": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-67892a7d-9e65-4e48-9832-3c57ae4ac7d1 req-c8bc6b48-7ae8-44c7-9aff-c7a18819c7ed service nova] Acquired lock "refresh_cache-8d533f5f-dbed-4878-a9b1-2ce4fae8fa13" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.network.neutron [req-67892a7d-9e65-4e48-9832-3c57ae4ac7d1 req-c8bc6b48-7ae8-44c7-9aff-c7a18819c7ed service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Refreshing network info cache for port 2ef93926-6cf3-4d57-be98-ab555ac58b4c {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Start _get_guest_xml network_info=[{"id": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "address": "fa:16:3e:e2:fd:44", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2ef93926-6c", "ovs_interfaceid": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:47:40 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:47:40 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:47:37Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1485022675',display_name='tempest-ServerActionsTestJSON-server-1485022675',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1485022675',id=36,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKmrI9wy1CUIwhlKuXvzk0mD7Pt5nZFvI1CsqS0do6Lk76JQOvGl5lvx9rkz7UAM9WGocGXyWSOUOuKTtYZRD2HqKUbTN4VPOoliE1yuu+s0lx7vuZu45Mj7wDb2S8Ksyw==',key_name='tempest-keypair-364276420',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='815b65aa5ad9442db0a087c646f96bbd',ramdisk_id='',reservation_id='r-5vy01cil',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestJSON-1898059859',owner_user_name='tempest-ServerActionsTestJSON-1898059859-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:47:39Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a7f422f1a05a4a92ba387e424850b360',uuid=8d533f5f-dbed-4878-a9b1-2ce4fae8fa13,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "address": "fa:16:3e:e2:fd:44", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2ef93926-6c", "ovs_interfaceid": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converting VIF {"id": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "address": "fa:16:3e:e2:fd:44", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2ef93926-6c", "ovs_interfaceid": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e2:fd:44,bridge_name='br-int',has_traffic_filtering=True,id=2ef93926-6cf3-4d57-be98-ab555ac58b4c,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2ef93926-6c') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.objects.instance [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'pci_devices' on Instance uuid 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:47:40 user nova-compute[72081]:   <uuid>8d533f5f-dbed-4878-a9b1-2ce4fae8fa13</uuid>
Apr 12 03:47:40 user nova-compute[72081]:   <name>instance-00000024</name>
Apr 12 03:47:40 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:47:40 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:47:40 user nova-compute[72081]:   <metadata>
Apr 12 03:47:40 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:47:40 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:47:40 user nova-compute[72081]:       <nova:name>tempest-ServerActionsTestJSON-server-1485022675</nova:name>
Apr 12 03:47:40 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:47:40</nova:creationTime>
Apr 12 03:47:40 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:47:40 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:47:40 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:47:40 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:47:40 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:47:40 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:47:40 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:47:40 user nova-compute[72081]:       <nova:owner>
Apr 12 03:47:40 user nova-compute[72081]:         <nova:user uuid="a7f422f1a05a4a92ba387e424850b360">tempest-ServerActionsTestJSON-1898059859-project-member</nova:user>
Apr 12 03:47:40 user nova-compute[72081]:         <nova:project uuid="815b65aa5ad9442db0a087c646f96bbd">tempest-ServerActionsTestJSON-1898059859</nova:project>
Apr 12 03:47:40 user nova-compute[72081]:       </nova:owner>
Apr 12 03:47:40 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:47:40 user nova-compute[72081]:       <nova:ports>
Apr 12 03:47:40 user nova-compute[72081]:         <nova:port uuid="2ef93926-6cf3-4d57-be98-ab555ac58b4c">
Apr 12 03:47:40 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 12 03:47:40 user nova-compute[72081]:         </nova:port>
Apr 12 03:47:40 user nova-compute[72081]:       </nova:ports>
Apr 12 03:47:40 user nova-compute[72081]:     </nova:instance>
Apr 12 03:47:40 user nova-compute[72081]:   </metadata>
Apr 12 03:47:40 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:47:40 user nova-compute[72081]:     <system>
Apr 12 03:47:40 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:47:40 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:47:40 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:47:40 user nova-compute[72081]:       <entry name="serial">8d533f5f-dbed-4878-a9b1-2ce4fae8fa13</entry>
Apr 12 03:47:40 user nova-compute[72081]:       <entry name="uuid">8d533f5f-dbed-4878-a9b1-2ce4fae8fa13</entry>
Apr 12 03:47:40 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:47:40 user nova-compute[72081]:     </system>
Apr 12 03:47:40 user nova-compute[72081]:   </sysinfo>
Apr 12 03:47:40 user nova-compute[72081]:   <os>
Apr 12 03:47:40 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:47:40 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:47:40 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:47:40 user nova-compute[72081]:   </os>
Apr 12 03:47:40 user nova-compute[72081]:   <features>
Apr 12 03:47:40 user nova-compute[72081]:     <acpi/>
Apr 12 03:47:40 user nova-compute[72081]:     <apic/>
Apr 12 03:47:40 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:47:40 user nova-compute[72081]:   </features>
Apr 12 03:47:40 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:47:40 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:47:40 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:47:40 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:47:40 user nova-compute[72081]:   </clock>
Apr 12 03:47:40 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:47:40 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:47:40 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:47:40 user nova-compute[72081]:   </cpu>
Apr 12 03:47:40 user nova-compute[72081]:   <devices>
Apr 12 03:47:40 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:47:40 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:47:40 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk"/>
Apr 12 03:47:40 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:47:40 user nova-compute[72081]:     </disk>
Apr 12 03:47:40 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:47:40 user nova-compute[72081]:       <mac address="fa:16:3e:e2:fd:44"/>
Apr 12 03:47:40 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:47:40 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:47:40 user nova-compute[72081]:       <target dev="tap2ef93926-6c"/>
Apr 12 03:47:40 user nova-compute[72081]:     </interface>
Apr 12 03:47:40 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:47:40 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/console.log" append="off"/>
Apr 12 03:47:40 user nova-compute[72081]:     </serial>
Apr 12 03:47:40 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:47:40 user nova-compute[72081]:     <video>
Apr 12 03:47:40 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:47:40 user nova-compute[72081]:     </video>
Apr 12 03:47:40 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:47:40 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:47:40 user nova-compute[72081]:     </rng>
Apr 12 03:47:40 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:47:40 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:47:40 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:47:40 user nova-compute[72081]:     </memballoon>
Apr 12 03:47:40 user nova-compute[72081]:   </devices>
Apr 12 03:47:40 user nova-compute[72081]: </domain>
Apr 12 03:47:40 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:47:37Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1485022675',display_name='tempest-ServerActionsTestJSON-server-1485022675',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1485022675',id=36,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKmrI9wy1CUIwhlKuXvzk0mD7Pt5nZFvI1CsqS0do6Lk76JQOvGl5lvx9rkz7UAM9WGocGXyWSOUOuKTtYZRD2HqKUbTN4VPOoliE1yuu+s0lx7vuZu45Mj7wDb2S8Ksyw==',key_name='tempest-keypair-364276420',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='815b65aa5ad9442db0a087c646f96bbd',ramdisk_id='',reservation_id='r-5vy01cil',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestJSON-1898059859',owner_user_name='tempest-ServerActionsTestJSON-1898059859-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:47:39Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a7f422f1a05a4a92ba387e424850b360',uuid=8d533f5f-dbed-4878-a9b1-2ce4fae8fa13,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "address": "fa:16:3e:e2:fd:44", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2ef93926-6c", "ovs_interfaceid": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converting VIF {"id": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "address": "fa:16:3e:e2:fd:44", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2ef93926-6c", "ovs_interfaceid": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e2:fd:44,bridge_name='br-int',has_traffic_filtering=True,id=2ef93926-6cf3-4d57-be98-ab555ac58b4c,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2ef93926-6c') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG os_vif [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:e2:fd:44,bridge_name='br-int',has_traffic_filtering=True,id=2ef93926-6cf3-4d57-be98-ab555ac58b4c,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2ef93926-6c') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap2ef93926-6c, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap2ef93926-6c, col_values=(('external_ids', {'iface-id': '2ef93926-6cf3-4d57-be98-ab555ac58b4c', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:e2:fd:44', 'vm-uuid': '8d533f5f-dbed-4878-a9b1-2ce4fae8fa13'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:40 user nova-compute[72081]: INFO os_vif [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:e2:fd:44,bridge_name='br-int',has_traffic_filtering=True,id=2ef93926-6cf3-4d57-be98-ab555ac58b4c,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2ef93926-6c')
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:47:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] No VIF found with MAC fa:16:3e:e2:fd:44, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:47:41 user nova-compute[72081]: DEBUG nova.network.neutron [req-67892a7d-9e65-4e48-9832-3c57ae4ac7d1 req-c8bc6b48-7ae8-44c7-9aff-c7a18819c7ed service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Updated VIF entry in instance network info cache for port 2ef93926-6cf3-4d57-be98-ab555ac58b4c. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:47:41 user nova-compute[72081]: DEBUG nova.network.neutron [req-67892a7d-9e65-4e48-9832-3c57ae4ac7d1 req-c8bc6b48-7ae8-44c7-9aff-c7a18819c7ed service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Updating instance_info_cache with network_info: [{"id": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "address": "fa:16:3e:e2:fd:44", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2ef93926-6c", "ovs_interfaceid": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:47:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-67892a7d-9e65-4e48-9832-3c57ae4ac7d1 req-c8bc6b48-7ae8-44c7-9aff-c7a18819c7ed service nova] Releasing lock "refresh_cache-8d533f5f-dbed-4878-a9b1-2ce4fae8fa13" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:47:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:42 user nova-compute[72081]: DEBUG nova.compute.manager [req-8ef9d16d-c553-495a-8f0b-e909dbd3561d req-18a5d985-2754-4f2f-a63e-69f924e8c574 service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Received event network-vif-plugged-2ef93926-6cf3-4d57-be98-ab555ac58b4c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:47:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8ef9d16d-c553-495a-8f0b-e909dbd3561d req-18a5d985-2754-4f2f-a63e-69f924e8c574 service nova] Acquiring lock "8d533f5f-dbed-4878-a9b1-2ce4fae8fa13-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8ef9d16d-c553-495a-8f0b-e909dbd3561d req-18a5d985-2754-4f2f-a63e-69f924e8c574 service nova] Lock "8d533f5f-dbed-4878-a9b1-2ce4fae8fa13-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-8ef9d16d-c553-495a-8f0b-e909dbd3561d req-18a5d985-2754-4f2f-a63e-69f924e8c574 service nova] Lock "8d533f5f-dbed-4878-a9b1-2ce4fae8fa13-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:42 user nova-compute[72081]: DEBUG nova.compute.manager [req-8ef9d16d-c553-495a-8f0b-e909dbd3561d req-18a5d985-2754-4f2f-a63e-69f924e8c574 service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] No waiting events found dispatching network-vif-plugged-2ef93926-6cf3-4d57-be98-ab555ac58b4c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:47:42 user nova-compute[72081]: WARNING nova.compute.manager [req-8ef9d16d-c553-495a-8f0b-e909dbd3561d req-18a5d985-2754-4f2f-a63e-69f924e8c574 service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Received unexpected event network-vif-plugged-2ef93926-6cf3-4d57-be98-ab555ac58b4c for instance with vm_state building and task_state spawning.
Apr 12 03:47:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:44 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271264.3102775, 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:47:44 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] VM Resumed (Lifecycle Event)
Apr 12 03:47:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:47:44 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:47:44 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Instance spawned successfully.
Apr 12 03:47:44 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:47:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:47:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:47:44 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:47:44 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:47:44 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:47:44 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:47:44 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:47:44 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:47:44 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:47:44 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271264.3104649, 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:47:44 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] VM Started (Lifecycle Event)
Apr 12 03:47:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:47:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:47:44 user nova-compute[72081]: INFO nova.compute.manager [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Took 5.45 seconds to spawn the instance on the hypervisor.
Apr 12 03:47:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:47:44 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:47:44 user nova-compute[72081]: DEBUG nova.compute.manager [req-ce206dd1-8df8-4ffc-882e-9a4b4efe9c38 req-02f8f60a-f2b1-4220-8605-fb7360a8c046 service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Received event network-vif-plugged-2ef93926-6cf3-4d57-be98-ab555ac58b4c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:47:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ce206dd1-8df8-4ffc-882e-9a4b4efe9c38 req-02f8f60a-f2b1-4220-8605-fb7360a8c046 service nova] Acquiring lock "8d533f5f-dbed-4878-a9b1-2ce4fae8fa13-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ce206dd1-8df8-4ffc-882e-9a4b4efe9c38 req-02f8f60a-f2b1-4220-8605-fb7360a8c046 service nova] Lock "8d533f5f-dbed-4878-a9b1-2ce4fae8fa13-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ce206dd1-8df8-4ffc-882e-9a4b4efe9c38 req-02f8f60a-f2b1-4220-8605-fb7360a8c046 service nova] Lock "8d533f5f-dbed-4878-a9b1-2ce4fae8fa13-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:44 user nova-compute[72081]: DEBUG nova.compute.manager [req-ce206dd1-8df8-4ffc-882e-9a4b4efe9c38 req-02f8f60a-f2b1-4220-8605-fb7360a8c046 service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] No waiting events found dispatching network-vif-plugged-2ef93926-6cf3-4d57-be98-ab555ac58b4c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:47:44 user nova-compute[72081]: WARNING nova.compute.manager [req-ce206dd1-8df8-4ffc-882e-9a4b4efe9c38 req-02f8f60a-f2b1-4220-8605-fb7360a8c046 service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Received unexpected event network-vif-plugged-2ef93926-6cf3-4d57-be98-ab555ac58b4c for instance with vm_state building and task_state spawning.
Apr 12 03:47:44 user nova-compute[72081]: INFO nova.compute.manager [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Took 6.10 seconds to build instance.
Apr 12 03:47:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f7f74d21-ca34-4f47-9e7a-c25ccda6f681 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "8d533f5f-dbed-4878-a9b1-2ce4fae8fa13" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.208s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:45 user nova-compute[72081]: INFO nova.compute.manager [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Detaching volume 188c90f2-90ef-4909-8917-56808fd9a068
Apr 12 03:47:45 user nova-compute[72081]: INFO nova.virt.block_device [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Attempting to driver detach volume 188c90f2-90ef-4909-8917-56808fd9a068 from mountpoint /dev/vdb
Apr 12 03:47:45 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Attempting to detach device vdb from instance 606425bb-8968-41f9-b3a5-80b6fd188a67 from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:47:45 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:47:45 user nova-compute[72081]:   <source dev="/dev/sdd"/>
Apr 12 03:47:45 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:47:45 user nova-compute[72081]:   <serial>188c90f2-90ef-4909-8917-56808fd9a068</serial>
Apr 12 03:47:45 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Apr 12 03:47:45 user nova-compute[72081]: </disk>
Apr 12 03:47:45 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:47:45 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Successfully detached device vdb from instance 606425bb-8968-41f9-b3a5-80b6fd188a67 from the persistent domain config.
Apr 12 03:47:45 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 606425bb-8968-41f9-b3a5-80b6fd188a67 from the live domain config. {{(pid=72081) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:47:45 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:47:45 user nova-compute[72081]:   <source dev="/dev/sdd"/>
Apr 12 03:47:45 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:47:45 user nova-compute[72081]:   <serial>188c90f2-90ef-4909-8917-56808fd9a068</serial>
Apr 12 03:47:45 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Apr 12 03:47:45 user nova-compute[72081]: </disk>
Apr 12 03:47:45 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Received event <DeviceRemovedEvent: 1681271265.6298623, 606425bb-8968-41f9-b3a5-80b6fd188a67 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72081) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 12 03:47:45 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Successfully detached device vdb from instance 606425bb-8968-41f9-b3a5-80b6fd188a67 from the live domain config.
Apr 12 03:47:45 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'VoqWQaSf', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:47:45 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:47:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[438e409c-d9f6-4ec4-bfcd-1e139f5ccb46]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:47:45 user nova-compute[72081]: No targets found.
Apr 12 03:47:45 user nova-compute[72081]: iSNS:
Apr 12 03:47:45 user nova-compute[72081]: No targets found.
Apr 12 03:47:45 user nova-compute[72081]: STATIC:
Apr 12 03:47:45 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27
Apr 12 03:47:45 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:47:45 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:47:45 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:47:45 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:47:45 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:47:45 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068
Apr 12 03:47:45 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:47:45 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:47:45 user nova-compute[72081]: FIRMWARE:
Apr 12 03:47:45 user nova-compute[72081]: No targets found.
Apr 12 03:47:45 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:47:45 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:47:45 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[39db494a-c0f4-4c89-a081-f00a9720aa25]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[10762e1d-3d17-41e2-a805-401257bcd3df]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:47:45 user nova-compute[72081]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:47:45 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:47:45 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:47:45 user nova-compute[72081]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 (non-flash)
Apr 12 03:47:45 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:47:45 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3f745f70a0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068'): ({'sdd'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Removing single pathed devices sdd {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8d12b6f4-18ab-455b-87b5-9c4a807a0e75]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a18f24a6-13cd-44f9-a4d5-3ebe90354e2d]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Flushing IO for device /dev/sdd {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.013s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1c1c96e6-def7-455b-9c2e-9a1a6716420b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.056s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[352348fb-ff1f-4338-9f03-1def40d4db8b]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] SCSI volumes sdd have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ff40f4cb-5da9-49fd-8559-c188d11fb264]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --logout" returned: 0 in 0.053s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[66466665-7eef-43d9-abba-105f43ebbb28]: (4, ('Logging out of session [sid: 36, target: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068, portal: 172.16.0.220,3260]\nLogout of [sid: 36, target: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 36, target: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068, portal: 172.16.0.220,3260]
Apr 12 03:47:45 user nova-compute[72081]: Logout of [sid: 36, target: iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068, portal: 172.16.0.220,3260] successful.
Apr 12 03:47:45 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:188c90f2-90ef-4909-8917-56808fd9a068 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[cce6ff3b-de10-4782-a3aa-2eb38f50cc78]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.296s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] <== disconnect_volume: return (297ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:47:45 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:47:46 user nova-compute[72081]: DEBUG nova.objects.instance [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'flavor' on Instance uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:47:46 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-412b0092-81f5-4a77-a76e-60d2309c3743 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.733s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f2e202f0-e149-465f-b021-dea70d131532 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67" by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f2e202f0-e149-465f-b021-dea70d131532 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67" acquired by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:47 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f2e202f0-e149-465f-b021-dea70d131532 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:47:47 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f2e202f0-e149-465f-b021-dea70d131532 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Stopping instance; current vm_state: active, current task_state: powering-off, current DB power_state: 1, current VM power_state: 1 {{(pid=72081) do_stop_instance /opt/stack/nova/nova/compute/manager.py:3323}}
Apr 12 03:47:47 user nova-compute[72081]: DEBUG nova.objects.instance [None req-f2e202f0-e149-465f-b021-dea70d131532 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'flavor' on Instance uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:47:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:47 user nova-compute[72081]: DEBUG nova.compute.manager [req-dd3f6330-227f-4226-b80a-a0e081c25625 req-5cde1505-ba22-4d43-be22-1d5eea95c6fd service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received event network-vif-unplugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:47:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-dd3f6330-227f-4226-b80a-a0e081c25625 req-5cde1505-ba22-4d43-be22-1d5eea95c6fd service nova] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-dd3f6330-227f-4226-b80a-a0e081c25625 req-5cde1505-ba22-4d43-be22-1d5eea95c6fd service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-dd3f6330-227f-4226-b80a-a0e081c25625 req-5cde1505-ba22-4d43-be22-1d5eea95c6fd service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:47 user nova-compute[72081]: DEBUG nova.compute.manager [req-dd3f6330-227f-4226-b80a-a0e081c25625 req-5cde1505-ba22-4d43-be22-1d5eea95c6fd service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] No waiting events found dispatching network-vif-unplugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:47:47 user nova-compute[72081]: WARNING nova.compute.manager [req-dd3f6330-227f-4226-b80a-a0e081c25625 req-5cde1505-ba22-4d43-be22-1d5eea95c6fd service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received unexpected event network-vif-unplugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 for instance with vm_state active and task_state powering-off.
Apr 12 03:47:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:48 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Instance destroyed successfully.
Apr 12 03:47:48 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f2e202f0-e149-465f-b021-dea70d131532 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:47:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:48 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f2e202f0-e149-465f-b021-dea70d131532 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67" "released" by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" :: held 0.882s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:49 user nova-compute[72081]: DEBUG nova.compute.manager [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:47:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:49 user nova-compute[72081]: DEBUG nova.compute.manager [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] No waiting events found dispatching network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:47:49 user nova-compute[72081]: WARNING nova.compute.manager [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received unexpected event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 for instance with vm_state stopped and task_state None.
Apr 12 03:47:49 user nova-compute[72081]: DEBUG nova.compute.manager [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:47:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:49 user nova-compute[72081]: DEBUG nova.compute.manager [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] No waiting events found dispatching network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:47:49 user nova-compute[72081]: WARNING nova.compute.manager [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received unexpected event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 for instance with vm_state stopped and task_state None.
Apr 12 03:47:49 user nova-compute[72081]: DEBUG nova.compute.manager [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:47:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:49 user nova-compute[72081]: DEBUG nova.compute.manager [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] No waiting events found dispatching network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:47:49 user nova-compute[72081]: WARNING nova.compute.manager [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received unexpected event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 for instance with vm_state stopped and task_state None.
Apr 12 03:47:49 user nova-compute[72081]: DEBUG nova.compute.manager [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received event network-vif-unplugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:47:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:49 user nova-compute[72081]: DEBUG nova.compute.manager [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] No waiting events found dispatching network-vif-unplugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:47:49 user nova-compute[72081]: WARNING nova.compute.manager [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received unexpected event network-vif-unplugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 for instance with vm_state stopped and task_state None.
Apr 12 03:47:49 user nova-compute[72081]: DEBUG nova.compute.manager [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:47:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:49 user nova-compute[72081]: DEBUG nova.compute.manager [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] No waiting events found dispatching network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:47:49 user nova-compute[72081]: WARNING nova.compute.manager [req-961d6cc8-c68b-4204-93d3-66683043f8cc req-2416816f-439a-4387-a39e-e6dbf8af47ab service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received unexpected event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 for instance with vm_state stopped and task_state None.
Apr 12 03:47:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b4b8a209-9059-45f8-a68a-65404b470faf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "a264f3f7-b0ea-482a-a438-909f06a80726" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b4b8a209-9059-45f8-a68a-65404b470faf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "a264f3f7-b0ea-482a-a438-909f06a80726" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b4b8a209-9059-45f8-a68a-65404b470faf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "a264f3f7-b0ea-482a-a438-909f06a80726-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b4b8a209-9059-45f8-a68a-65404b470faf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "a264f3f7-b0ea-482a-a438-909f06a80726-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b4b8a209-9059-45f8-a68a-65404b470faf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "a264f3f7-b0ea-482a-a438-909f06a80726-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:57 user nova-compute[72081]: INFO nova.compute.manager [None req-b4b8a209-9059-45f8-a68a-65404b470faf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Terminating instance
Apr 12 03:47:57 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b4b8a209-9059-45f8-a68a-65404b470faf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:47:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:57 user nova-compute[72081]: DEBUG nova.compute.manager [req-34d0c65b-0795-4532-b443-cfd6446f40fc req-de06af69-a6b4-4a1b-a571-f46ea556d5da service nova] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Received event network-vif-unplugged-e294fdef-5e1a-47fd-bcb1-08b22b4941b0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:47:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-34d0c65b-0795-4532-b443-cfd6446f40fc req-de06af69-a6b4-4a1b-a571-f46ea556d5da service nova] Acquiring lock "a264f3f7-b0ea-482a-a438-909f06a80726-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-34d0c65b-0795-4532-b443-cfd6446f40fc req-de06af69-a6b4-4a1b-a571-f46ea556d5da service nova] Lock "a264f3f7-b0ea-482a-a438-909f06a80726-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-34d0c65b-0795-4532-b443-cfd6446f40fc req-de06af69-a6b4-4a1b-a571-f46ea556d5da service nova] Lock "a264f3f7-b0ea-482a-a438-909f06a80726-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:57 user nova-compute[72081]: DEBUG nova.compute.manager [req-34d0c65b-0795-4532-b443-cfd6446f40fc req-de06af69-a6b4-4a1b-a571-f46ea556d5da service nova] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] No waiting events found dispatching network-vif-unplugged-e294fdef-5e1a-47fd-bcb1-08b22b4941b0 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:47:57 user nova-compute[72081]: DEBUG nova.compute.manager [req-34d0c65b-0795-4532-b443-cfd6446f40fc req-de06af69-a6b4-4a1b-a571-f46ea556d5da service nova] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Received event network-vif-unplugged-e294fdef-5e1a-47fd-bcb1-08b22b4941b0 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:47:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:58 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Instance destroyed successfully.
Apr 12 03:47:58 user nova-compute[72081]: DEBUG nova.objects.instance [None req-b4b8a209-9059-45f8-a68a-65404b470faf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lazy-loading 'resources' on Instance uuid a264f3f7-b0ea-482a-a438-909f06a80726 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:47:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-b4b8a209-9059-45f8-a68a-65404b470faf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:45:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-1427168436',display_name='tempest-AttachVolumeNegativeTest-server-1427168436',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-1427168436',id=33,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEWW3PcC6A3oaeI+T5mnA66uGmCFTzs3IbsSTkdd3DCArM0rR6yU/WOLMnATsGubN5kt/JcbJyDd6J/OXjO1xRs8Gm7pg5070UpCpDFnH5m0xDJi0dRMwI1KMt4s8ojMvQ==',key_name='tempest-keypair-863037271',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:45:13Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='d13546149a4e44e1b782b99e9eb452e8',ramdisk_id='',reservation_id='r-3fhwenfs',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeNegativeTest-494105310',owner_user_name='tempest-AttachVolumeNegativeTest-494105310-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:45:14Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d52637cca2114e5e99e688ee371c835a',uuid=a264f3f7-b0ea-482a-a438-909f06a80726,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e294fdef-5e1a-47fd-bcb1-08b22b4941b0", "address": "fa:16:3e:4a:45:5f", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.163", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape294fdef-5e", "ovs_interfaceid": "e294fdef-5e1a-47fd-bcb1-08b22b4941b0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:47:58 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b4b8a209-9059-45f8-a68a-65404b470faf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Converting VIF {"id": "e294fdef-5e1a-47fd-bcb1-08b22b4941b0", "address": "fa:16:3e:4a:45:5f", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.163", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape294fdef-5e", "ovs_interfaceid": "e294fdef-5e1a-47fd-bcb1-08b22b4941b0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:47:58 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b4b8a209-9059-45f8-a68a-65404b470faf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:4a:45:5f,bridge_name='br-int',has_traffic_filtering=True,id=e294fdef-5e1a-47fd-bcb1-08b22b4941b0,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape294fdef-5e') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:47:58 user nova-compute[72081]: DEBUG os_vif [None req-b4b8a209-9059-45f8-a68a-65404b470faf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:4a:45:5f,bridge_name='br-int',has_traffic_filtering=True,id=e294fdef-5e1a-47fd-bcb1-08b22b4941b0,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape294fdef-5e') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:47:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tape294fdef-5e, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:47:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:47:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:58 user nova-compute[72081]: INFO os_vif [None req-b4b8a209-9059-45f8-a68a-65404b470faf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:4a:45:5f,bridge_name='br-int',has_traffic_filtering=True,id=e294fdef-5e1a-47fd-bcb1-08b22b4941b0,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape294fdef-5e')
Apr 12 03:47:58 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-b4b8a209-9059-45f8-a68a-65404b470faf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Deleting instance files /opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726_del
Apr 12 03:47:58 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-b4b8a209-9059-45f8-a68a-65404b470faf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Deletion of /opt/stack/data/nova/instances/a264f3f7-b0ea-482a-a438-909f06a80726_del complete
Apr 12 03:47:58 user nova-compute[72081]: INFO nova.compute.manager [None req-b4b8a209-9059-45f8-a68a-65404b470faf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Took 0.88 seconds to destroy the instance on the hypervisor.
Apr 12 03:47:58 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-b4b8a209-9059-45f8-a68a-65404b470faf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:47:58 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:47:58 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:47:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:47:58 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:47:58 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Took 0.72 seconds to deallocate network for instance.
Apr 12 03:47:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b4b8a209-9059-45f8-a68a-65404b470faf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b4b8a209-9059-45f8-a68a-65404b470faf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:59 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-b4b8a209-9059-45f8-a68a-65404b470faf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:47:59 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-b4b8a209-9059-45f8-a68a-65404b470faf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:47:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b4b8a209-9059-45f8-a68a-65404b470faf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.279s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:59 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-b4b8a209-9059-45f8-a68a-65404b470faf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Deleted allocations for instance a264f3f7-b0ea-482a-a438-909f06a80726
Apr 12 03:47:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b4b8a209-9059-45f8-a68a-65404b470faf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "a264f3f7-b0ea-482a-a438-909f06a80726" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.053s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-0fca9a74-333e-4efd-b8d0-3e2e07f669c8 req-1676a8fe-11e1-406b-bc4b-32ef47be3ef1 service nova] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Received event network-vif-plugged-e294fdef-5e1a-47fd-bcb1-08b22b4941b0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:47:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0fca9a74-333e-4efd-b8d0-3e2e07f669c8 req-1676a8fe-11e1-406b-bc4b-32ef47be3ef1 service nova] Acquiring lock "a264f3f7-b0ea-482a-a438-909f06a80726-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:47:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0fca9a74-333e-4efd-b8d0-3e2e07f669c8 req-1676a8fe-11e1-406b-bc4b-32ef47be3ef1 service nova] Lock "a264f3f7-b0ea-482a-a438-909f06a80726-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:47:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0fca9a74-333e-4efd-b8d0-3e2e07f669c8 req-1676a8fe-11e1-406b-bc4b-32ef47be3ef1 service nova] Lock "a264f3f7-b0ea-482a-a438-909f06a80726-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:47:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-0fca9a74-333e-4efd-b8d0-3e2e07f669c8 req-1676a8fe-11e1-406b-bc4b-32ef47be3ef1 service nova] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] No waiting events found dispatching network-vif-plugged-e294fdef-5e1a-47fd-bcb1-08b22b4941b0 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:47:59 user nova-compute[72081]: WARNING nova.compute.manager [req-0fca9a74-333e-4efd-b8d0-3e2e07f669c8 req-1676a8fe-11e1-406b-bc4b-32ef47be3ef1 service nova] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Received unexpected event network-vif-plugged-e294fdef-5e1a-47fd-bcb1-08b22b4941b0 for instance with vm_state deleted and task_state None.
Apr 12 03:47:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-0fca9a74-333e-4efd-b8d0-3e2e07f669c8 req-1676a8fe-11e1-406b-bc4b-32ef47be3ef1 service nova] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Received event network-vif-deleted-e294fdef-5e1a-47fd-bcb1-08b22b4941b0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:48:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:03 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681271268.3499343, 606425bb-8968-41f9-b3a5-80b6fd188a67 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:48:03 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] VM Stopped (Lifecycle Event)
Apr 12 03:48:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b2ac946c-17f4-4338-9095-77697ebd381a None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:48:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b2ac946c-17f4-4338-9095-77697ebd381a None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: stopped, current task_state: None, current DB power_state: 4, VM power_state: 4 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:48:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-be21f625-d1c0-4206-bf5a-5e24614f1b78 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "3d4407f4-e078-4777-a110-cb3f922be2e8" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-be21f625-d1c0-4206-bf5a-5e24614f1b78 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "3d4407f4-e078-4777-a110-cb3f922be2e8" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-be21f625-d1c0-4206-bf5a-5e24614f1b78 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "3d4407f4-e078-4777-a110-cb3f922be2e8-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-be21f625-d1c0-4206-bf5a-5e24614f1b78 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "3d4407f4-e078-4777-a110-cb3f922be2e8-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-be21f625-d1c0-4206-bf5a-5e24614f1b78 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "3d4407f4-e078-4777-a110-cb3f922be2e8-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:04 user nova-compute[72081]: INFO nova.compute.manager [None req-be21f625-d1c0-4206-bf5a-5e24614f1b78 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Terminating instance
Apr 12 03:48:04 user nova-compute[72081]: DEBUG nova.compute.manager [None req-be21f625-d1c0-4206-bf5a-5e24614f1b78 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:48:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-c2990030-c6f0-449a-b73c-36bbe1ad4278 req-3e7d7e9d-ecfa-4835-ada2-c1d533bbde2c service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Received event network-changed-e1586e63-ee44-4717-ad47-886b1c3f01d0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:48:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-c2990030-c6f0-449a-b73c-36bbe1ad4278 req-3e7d7e9d-ecfa-4835-ada2-c1d533bbde2c service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Refreshing instance network info cache due to event network-changed-e1586e63-ee44-4717-ad47-886b1c3f01d0. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:48:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c2990030-c6f0-449a-b73c-36bbe1ad4278 req-3e7d7e9d-ecfa-4835-ada2-c1d533bbde2c service nova] Acquiring lock "refresh_cache-12351dbd-0f2e-40a6-a10a-f40f5b6250ef" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:48:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c2990030-c6f0-449a-b73c-36bbe1ad4278 req-3e7d7e9d-ecfa-4835-ada2-c1d533bbde2c service nova] Acquired lock "refresh_cache-12351dbd-0f2e-40a6-a10a-f40f5b6250ef" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:48:04 user nova-compute[72081]: DEBUG nova.network.neutron [req-c2990030-c6f0-449a-b73c-36bbe1ad4278 req-3e7d7e9d-ecfa-4835-ada2-c1d533bbde2c service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Refreshing network info cache for port e1586e63-ee44-4717-ad47-886b1c3f01d0 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:48:05 user nova-compute[72081]: DEBUG nova.compute.manager [req-cbc0e0b4-40b8-4655-9f43-99c45781f4e7 req-b9e409d7-d5ce-4e09-8f03-76b7d04f1c9a service nova] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Received event network-vif-unplugged-aec6d617-eb2b-42b3-8cce-f21de7c1b91c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:48:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cbc0e0b4-40b8-4655-9f43-99c45781f4e7 req-b9e409d7-d5ce-4e09-8f03-76b7d04f1c9a service nova] Acquiring lock "3d4407f4-e078-4777-a110-cb3f922be2e8-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cbc0e0b4-40b8-4655-9f43-99c45781f4e7 req-b9e409d7-d5ce-4e09-8f03-76b7d04f1c9a service nova] Lock "3d4407f4-e078-4777-a110-cb3f922be2e8-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-cbc0e0b4-40b8-4655-9f43-99c45781f4e7 req-b9e409d7-d5ce-4e09-8f03-76b7d04f1c9a service nova] Lock "3d4407f4-e078-4777-a110-cb3f922be2e8-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:05 user nova-compute[72081]: DEBUG nova.compute.manager [req-cbc0e0b4-40b8-4655-9f43-99c45781f4e7 req-b9e409d7-d5ce-4e09-8f03-76b7d04f1c9a service nova] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] No waiting events found dispatching network-vif-unplugged-aec6d617-eb2b-42b3-8cce-f21de7c1b91c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:48:05 user nova-compute[72081]: DEBUG nova.compute.manager [req-cbc0e0b4-40b8-4655-9f43-99c45781f4e7 req-b9e409d7-d5ce-4e09-8f03-76b7d04f1c9a service nova] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Received event network-vif-unplugged-aec6d617-eb2b-42b3-8cce-f21de7c1b91c for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:48:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:05 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Instance destroyed successfully.
Apr 12 03:48:05 user nova-compute[72081]: DEBUG nova.objects.instance [None req-be21f625-d1c0-4206-bf5a-5e24614f1b78 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lazy-loading 'resources' on Instance uuid 3d4407f4-e078-4777-a110-cb3f922be2e8 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:48:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-be21f625-d1c0-4206-bf5a-5e24614f1b78 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:38:42Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-752552685',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-752552685',id=24,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:38:50Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='6e1fa352a8c144a1aada7ce0d8818429',ramdisk_id='',reservation_id='r-tlziajet',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-590302458',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:40:40Z,user_data=None,user_id='d99be569057f432484e4a60b46882bbd',uuid=3d4407f4-e078-4777-a110-cb3f922be2e8,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "aec6d617-eb2b-42b3-8cce-f21de7c1b91c", "address": "fa:16:3e:92:a6:05", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapaec6d617-eb", "ovs_interfaceid": "aec6d617-eb2b-42b3-8cce-f21de7c1b91c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:48:05 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-be21f625-d1c0-4206-bf5a-5e24614f1b78 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converting VIF {"id": "aec6d617-eb2b-42b3-8cce-f21de7c1b91c", "address": "fa:16:3e:92:a6:05", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapaec6d617-eb", "ovs_interfaceid": "aec6d617-eb2b-42b3-8cce-f21de7c1b91c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:48:05 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-be21f625-d1c0-4206-bf5a-5e24614f1b78 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:92:a6:05,bridge_name='br-int',has_traffic_filtering=True,id=aec6d617-eb2b-42b3-8cce-f21de7c1b91c,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapaec6d617-eb') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:48:05 user nova-compute[72081]: DEBUG os_vif [None req-be21f625-d1c0-4206-bf5a-5e24614f1b78 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:92:a6:05,bridge_name='br-int',has_traffic_filtering=True,id=aec6d617-eb2b-42b3-8cce-f21de7c1b91c,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapaec6d617-eb') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:48:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapaec6d617-eb, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:48:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:48:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:05 user nova-compute[72081]: INFO os_vif [None req-be21f625-d1c0-4206-bf5a-5e24614f1b78 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:92:a6:05,bridge_name='br-int',has_traffic_filtering=True,id=aec6d617-eb2b-42b3-8cce-f21de7c1b91c,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapaec6d617-eb')
Apr 12 03:48:05 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-be21f625-d1c0-4206-bf5a-5e24614f1b78 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Deleting instance files /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8_del
Apr 12 03:48:05 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-be21f625-d1c0-4206-bf5a-5e24614f1b78 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Deletion of /opt/stack/data/nova/instances/3d4407f4-e078-4777-a110-cb3f922be2e8_del complete
Apr 12 03:48:05 user nova-compute[72081]: INFO nova.compute.manager [None req-be21f625-d1c0-4206-bf5a-5e24614f1b78 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Took 0.85 seconds to destroy the instance on the hypervisor.
Apr 12 03:48:05 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-be21f625-d1c0-4206-bf5a-5e24614f1b78 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:48:05 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:48:05 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:48:06 user nova-compute[72081]: DEBUG nova.network.neutron [req-c2990030-c6f0-449a-b73c-36bbe1ad4278 req-3e7d7e9d-ecfa-4835-ada2-c1d533bbde2c service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Updated VIF entry in instance network info cache for port e1586e63-ee44-4717-ad47-886b1c3f01d0. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:48:06 user nova-compute[72081]: DEBUG nova.network.neutron [req-c2990030-c6f0-449a-b73c-36bbe1ad4278 req-3e7d7e9d-ecfa-4835-ada2-c1d533bbde2c service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Updating instance_info_cache with network_info: [{"id": "e1586e63-ee44-4717-ad47-886b1c3f01d0", "address": "fa:16:3e:eb:02:66", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.174", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1586e63-ee", "ovs_interfaceid": "e1586e63-ee44-4717-ad47-886b1c3f01d0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:48:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c2990030-c6f0-449a-b73c-36bbe1ad4278 req-3e7d7e9d-ecfa-4835-ada2-c1d533bbde2c service nova] Releasing lock "refresh_cache-12351dbd-0f2e-40a6-a10a-f40f5b6250ef" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:48:06 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:48:06 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Took 0.71 seconds to deallocate network for instance.
Apr 12 03:48:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-be21f625-d1c0-4206-bf5a-5e24614f1b78 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-be21f625-d1c0-4206-bf5a-5e24614f1b78 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:06 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-be21f625-d1c0-4206-bf5a-5e24614f1b78 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:48:06 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-be21f625-d1c0-4206-bf5a-5e24614f1b78 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:48:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-be21f625-d1c0-4206-bf5a-5e24614f1b78 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.263s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:06 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-be21f625-d1c0-4206-bf5a-5e24614f1b78 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Deleted allocations for instance 3d4407f4-e078-4777-a110-cb3f922be2e8
Apr 12 03:48:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-be21f625-d1c0-4206-bf5a-5e24614f1b78 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "3d4407f4-e078-4777-a110-cb3f922be2e8" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.999s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-b9343280-d718-4e98-b21e-966d65d4e81a req-1c4b127a-8f63-4777-99ae-d55016520070 service nova] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Received event network-vif-plugged-aec6d617-eb2b-42b3-8cce-f21de7c1b91c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:48:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b9343280-d718-4e98-b21e-966d65d4e81a req-1c4b127a-8f63-4777-99ae-d55016520070 service nova] Acquiring lock "3d4407f4-e078-4777-a110-cb3f922be2e8-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b9343280-d718-4e98-b21e-966d65d4e81a req-1c4b127a-8f63-4777-99ae-d55016520070 service nova] Lock "3d4407f4-e078-4777-a110-cb3f922be2e8-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b9343280-d718-4e98-b21e-966d65d4e81a req-1c4b127a-8f63-4777-99ae-d55016520070 service nova] Lock "3d4407f4-e078-4777-a110-cb3f922be2e8-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-b9343280-d718-4e98-b21e-966d65d4e81a req-1c4b127a-8f63-4777-99ae-d55016520070 service nova] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] No waiting events found dispatching network-vif-plugged-aec6d617-eb2b-42b3-8cce-f21de7c1b91c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:48:07 user nova-compute[72081]: WARNING nova.compute.manager [req-b9343280-d718-4e98-b21e-966d65d4e81a req-1c4b127a-8f63-4777-99ae-d55016520070 service nova] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Received unexpected event network-vif-plugged-aec6d617-eb2b-42b3-8cce-f21de7c1b91c for instance with vm_state deleted and task_state None.
Apr 12 03:48:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-b9343280-d718-4e98-b21e-966d65d4e81a req-1c4b127a-8f63-4777-99ae-d55016520070 service nova] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Received event network-vif-deleted-aec6d617-eb2b-42b3-8cce-f21de7c1b91c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:48:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "6e8d7159-e7c4-4dbd-ab06-074b1d4e3873" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "6e8d7159-e7c4-4dbd-ab06-074b1d4e3873" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:48:10 user nova-compute[72081]: INFO nova.compute.claims [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Claim successful on node user
Apr 12 03:48:10 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.404s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG nova.network.neutron [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:48:10 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:48:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:48:10 user nova-compute[72081]: INFO nova.virt.block_device [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Booting with volume b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 at /dev/vda
Apr 12 03:48:10 user nova-compute[72081]: DEBUG nova.policy [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'b00b348933224558aad1cb12921f100f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '003688353e2f4b36831c574ebaf65173', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG os_brick.utils [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[512cbc39-e7d6-4d15-a348-be9730c30a81]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4725674f-af1e-4b62-ae68-ce4a17ec2f51]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ea63d3e3-d31d-49eb-a36d-a32ff39db55e]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3b7d69c2-a0e5-44bb-9cf8-d02762653b1a]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:48:10 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:48:10 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ae6f9d10-1060-4cae-a486-2e7059a94f9c]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG os_brick.utils [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] <== get_connector_properties: return (118ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:48:10 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Updating existing volume attachment record: bb248f98-57b3-4ddb-9647-8d7ca6fa0937 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:48:11 user nova-compute[72081]: DEBUG nova.network.neutron [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Successfully created port: c06ddd73-9521-41d8-a907-7af844b2adac {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:48:12 user nova-compute[72081]: DEBUG nova.network.neutron [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Successfully updated port: c06ddd73-9521-41d8-a907-7af844b2adac {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:48:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "refresh_cache-6e8d7159-e7c4-4dbd-ab06-074b1d4e3873" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:48:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquired lock "refresh_cache-6e8d7159-e7c4-4dbd-ab06-074b1d4e3873" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:48:12 user nova-compute[72081]: DEBUG nova.network.neutron [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:48:12 user nova-compute[72081]: DEBUG nova.compute.manager [req-c652e50b-0ad9-4361-aa7e-06f2cda1f154 req-d821c51c-027c-480c-a5b2-79864341e490 service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Received event network-changed-c06ddd73-9521-41d8-a907-7af844b2adac {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:48:12 user nova-compute[72081]: DEBUG nova.compute.manager [req-c652e50b-0ad9-4361-aa7e-06f2cda1f154 req-d821c51c-027c-480c-a5b2-79864341e490 service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Refreshing instance network info cache due to event network-changed-c06ddd73-9521-41d8-a907-7af844b2adac. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:48:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c652e50b-0ad9-4361-aa7e-06f2cda1f154 req-d821c51c-027c-480c-a5b2-79864341e490 service nova] Acquiring lock "refresh_cache-6e8d7159-e7c4-4dbd-ab06-074b1d4e3873" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:48:12 user nova-compute[72081]: DEBUG nova.network.neutron [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:48:12 user nova-compute[72081]: DEBUG nova.network.neutron [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Updating instance_info_cache with network_info: [{"id": "c06ddd73-9521-41d8-a907-7af844b2adac", "address": "fa:16:3e:89:fe:e4", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc06ddd73-95", "ovs_interfaceid": "c06ddd73-9521-41d8-a907-7af844b2adac", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:48:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Releasing lock "refresh_cache-6e8d7159-e7c4-4dbd-ab06-074b1d4e3873" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:48:12 user nova-compute[72081]: DEBUG nova.compute.manager [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Instance network_info: |[{"id": "c06ddd73-9521-41d8-a907-7af844b2adac", "address": "fa:16:3e:89:fe:e4", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc06ddd73-95", "ovs_interfaceid": "c06ddd73-9521-41d8-a907-7af844b2adac", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:48:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c652e50b-0ad9-4361-aa7e-06f2cda1f154 req-d821c51c-027c-480c-a5b2-79864341e490 service nova] Acquired lock "refresh_cache-6e8d7159-e7c4-4dbd-ab06-074b1d4e3873" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:48:12 user nova-compute[72081]: DEBUG nova.network.neutron [req-c652e50b-0ad9-4361-aa7e-06f2cda1f154 req-d821c51c-027c-480c-a5b2-79864341e490 service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Refreshing network info cache for port c06ddd73-9521-41d8-a907-7af844b2adac {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:48:12 user nova-compute[72081]: DEBUG nova.network.neutron [req-c652e50b-0ad9-4361-aa7e-06f2cda1f154 req-d821c51c-027c-480c-a5b2-79864341e490 service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Updated VIF entry in instance network info cache for port c06ddd73-9521-41d8-a907-7af844b2adac. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:48:12 user nova-compute[72081]: DEBUG nova.network.neutron [req-c652e50b-0ad9-4361-aa7e-06f2cda1f154 req-d821c51c-027c-480c-a5b2-79864341e490 service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Updating instance_info_cache with network_info: [{"id": "c06ddd73-9521-41d8-a907-7af844b2adac", "address": "fa:16:3e:89:fe:e4", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc06ddd73-95", "ovs_interfaceid": "c06ddd73-9521-41d8-a907-7af844b2adac", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:48:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c652e50b-0ad9-4361-aa7e-06f2cda1f154 req-d821c51c-027c-480c-a5b2-79864341e490 service nova] Releasing lock "refresh_cache-6e8d7159-e7c4-4dbd-ab06-074b1d4e3873" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:48:12 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681271277.996833, a264f3f7-b0ea-482a-a438-909f06a80726 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:48:12 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] VM Stopped (Lifecycle Event)
Apr 12 03:48:13 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b9db9e55-06c4-46f4-997a-3e37955aae51 None None] [instance: a264f3f7-b0ea-482a-a438-909f06a80726] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:48:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:14 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:48:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG nova.compute.manager [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:48:15 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Creating image(s)
Apr 12 03:48:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Did not create local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Ensure instance console log exists: /opt/stack/data/nova/instances/6e8d7159-e7c4-4dbd-ab06-074b1d4e3873/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Start _get_guest_xml network_info=[{"id": "c06ddd73-9521-41d8-a907-7af844b2adac", "address": "fa:16:3e:89:fe:e4", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc06ddd73-95", "ovs_interfaceid": "c06ddd73-9521-41d8-a907-7af844b2adac", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mDcBFWhK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '6e8d7159-e7c4-4dbd-ab06-074b1d4e3873', 'attached_at': '', 'detached_at': '', 'volume_id': 'b27d5f9c-2ca3-4c1d-86b8-9af733bddd76', 'serial': 'b27d5f9c-2ca3-4c1d-86b8-9af733bddd76'}, 'disk_bus': 'virtio', 'delete_on_termination': False, 'boot_index': 0, 'guest_format': None, 'device_type': 'disk', 'attachment_id': 'bb248f98-57b3-4ddb-9647-8d7ca6fa0937', 'volume_type': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:48:15 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:48:15 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:48:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mDcBFWhK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:48:15 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:48:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:48:15 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:48:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:48:15 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[00c2e291-f32d-40fe-9da5-a5ff54da0330]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:48:15 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5c02502-85b6-4c13-aecb-b3805c94c7e0]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76] added
Apr 12 03:48:15 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0311eafa-921f-4a4d-a925-d61cf32fe092]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4d8d83c3-6406-4902-9cd1-8743248f63a4]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v mDcBFWhK {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v mDcBFWhK" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[87e93d02-a99c-4b0c-b1b9-43d6cd6c0445]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'mDcBFWhK'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1466d407-f4d0-40e5-bf65-74a3a6fbe2b5]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[381f1fb2-37fe-4da1-8256-e2c0e179b9ff]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:48:15 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:48:15 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:48:15 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:48:15 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 -p 172.16.0.220:3260 --login" returned: 0 in 0.058s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[80ef3382-3f08-4d89-9fa1-fbf413d3f2ff]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76, portal: 172.16.0.220,3260]
Apr 12 03:48:15 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76, portal: 172.16.0.220,3260] successful.
Apr 12 03:48:15 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.029s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[26b77ea5-f3ed-43a2-a0c1-cc5fe1ea5f61]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b7c17521-4883-44a5-a9c5-8b92cbec685c]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:48:15 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:48:15 user nova-compute[72081]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 (non-flash)
Apr 12 03:48:15 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:48:15 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:48:15 user nova-compute[72081]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 (non-flash)
Apr 12 03:48:15 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.284s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] HCTL ('34', '-', '-', 0) found on session 40 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[724b5901-14df-4c96-bad9-e09e5824ebfd]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Searching for a device in session 40 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mDcBFWhK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:15 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:48:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.317s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:48:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] <== connect_volume: return (1319ms) {'type': 'block', 'scsi_wwn': '23664303066613765', 'path': '/dev/sdc'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:48:16 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23664303066613765', 'path': '/dev/sdc'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:48:16 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:48:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-682928462',display_name='tempest-TestVolumeBootPattern-server-682928462',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-682928462',id=37,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBASnyncVOhHKPzClYx3zwpDcJrieJkBwHOEHWlg2NFTJJdBa1/TdotUOLxC4yYnCsrHxIQy7r8qgZsehiquLQNSYVTnDsRU6DmQUCbMUYEphptEyPo7yf2/NkfI88nssBw==',key_name='tempest-TestVolumeBootPattern-54080024',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='003688353e2f4b36831c574ebaf65173',ramdisk_id='',reservation_id='r-ioaiilyi',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-235043461',owner_user_name='tempest-TestVolumeBootPattern-235043461-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:48:11Z,user_data=None,user_id='b00b348933224558aad1cb12921f100f',uuid=6e8d7159-e7c4-4dbd-ab06-074b1d4e3873,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c06ddd73-9521-41d8-a907-7af844b2adac", "address": "fa:16:3e:89:fe:e4", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc06ddd73-95", "ovs_interfaceid": "c06ddd73-9521-41d8-a907-7af844b2adac", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:48:16 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converting VIF {"id": "c06ddd73-9521-41d8-a907-7af844b2adac", "address": "fa:16:3e:89:fe:e4", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc06ddd73-95", "ovs_interfaceid": "c06ddd73-9521-41d8-a907-7af844b2adac", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:48:16 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:89:fe:e4,bridge_name='br-int',has_traffic_filtering=True,id=c06ddd73-9521-41d8-a907-7af844b2adac,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc06ddd73-95') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:48:16 user nova-compute[72081]: DEBUG nova.objects.instance [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lazy-loading 'pci_devices' on Instance uuid 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:48:16 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:48:16 user nova-compute[72081]:   <uuid>6e8d7159-e7c4-4dbd-ab06-074b1d4e3873</uuid>
Apr 12 03:48:16 user nova-compute[72081]:   <name>instance-00000025</name>
Apr 12 03:48:16 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:48:16 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:48:16 user nova-compute[72081]:   <metadata>
Apr 12 03:48:16 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:48:16 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:48:16 user nova-compute[72081]:       <nova:name>tempest-TestVolumeBootPattern-server-682928462</nova:name>
Apr 12 03:48:16 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:48:15</nova:creationTime>
Apr 12 03:48:16 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:48:16 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:48:16 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:48:16 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:48:16 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:48:16 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:48:16 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:48:16 user nova-compute[72081]:       <nova:owner>
Apr 12 03:48:16 user nova-compute[72081]:         <nova:user uuid="b00b348933224558aad1cb12921f100f">tempest-TestVolumeBootPattern-235043461-project-member</nova:user>
Apr 12 03:48:16 user nova-compute[72081]:         <nova:project uuid="003688353e2f4b36831c574ebaf65173">tempest-TestVolumeBootPattern-235043461</nova:project>
Apr 12 03:48:16 user nova-compute[72081]:       </nova:owner>
Apr 12 03:48:16 user nova-compute[72081]:       <nova:ports>
Apr 12 03:48:16 user nova-compute[72081]:         <nova:port uuid="c06ddd73-9521-41d8-a907-7af844b2adac">
Apr 12 03:48:16 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.7" ipVersion="4"/>
Apr 12 03:48:16 user nova-compute[72081]:         </nova:port>
Apr 12 03:48:16 user nova-compute[72081]:       </nova:ports>
Apr 12 03:48:16 user nova-compute[72081]:     </nova:instance>
Apr 12 03:48:16 user nova-compute[72081]:   </metadata>
Apr 12 03:48:16 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:48:16 user nova-compute[72081]:     <system>
Apr 12 03:48:16 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:48:16 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:48:16 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:48:16 user nova-compute[72081]:       <entry name="serial">6e8d7159-e7c4-4dbd-ab06-074b1d4e3873</entry>
Apr 12 03:48:16 user nova-compute[72081]:       <entry name="uuid">6e8d7159-e7c4-4dbd-ab06-074b1d4e3873</entry>
Apr 12 03:48:16 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:48:16 user nova-compute[72081]:     </system>
Apr 12 03:48:16 user nova-compute[72081]:   </sysinfo>
Apr 12 03:48:16 user nova-compute[72081]:   <os>
Apr 12 03:48:16 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:48:16 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:48:16 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:48:16 user nova-compute[72081]:   </os>
Apr 12 03:48:16 user nova-compute[72081]:   <features>
Apr 12 03:48:16 user nova-compute[72081]:     <acpi/>
Apr 12 03:48:16 user nova-compute[72081]:     <apic/>
Apr 12 03:48:16 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:48:16 user nova-compute[72081]:   </features>
Apr 12 03:48:16 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:48:16 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:48:16 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:48:16 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:48:16 user nova-compute[72081]:   </clock>
Apr 12 03:48:16 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:48:16 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:48:16 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:48:16 user nova-compute[72081]:   </cpu>
Apr 12 03:48:16 user nova-compute[72081]:   <devices>
Apr 12 03:48:16 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:48:16 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:48:16 user nova-compute[72081]:       <source dev="/dev/sdc"/>
Apr 12 03:48:16 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:48:16 user nova-compute[72081]:       <serial>b27d5f9c-2ca3-4c1d-86b8-9af733bddd76</serial>
Apr 12 03:48:16 user nova-compute[72081]:     </disk>
Apr 12 03:48:16 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:48:16 user nova-compute[72081]:       <mac address="fa:16:3e:89:fe:e4"/>
Apr 12 03:48:16 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:48:16 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:48:16 user nova-compute[72081]:       <target dev="tapc06ddd73-95"/>
Apr 12 03:48:16 user nova-compute[72081]:     </interface>
Apr 12 03:48:16 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:48:16 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/6e8d7159-e7c4-4dbd-ab06-074b1d4e3873/console.log" append="off"/>
Apr 12 03:48:16 user nova-compute[72081]:     </serial>
Apr 12 03:48:16 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:48:16 user nova-compute[72081]:     <video>
Apr 12 03:48:16 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:48:16 user nova-compute[72081]:     </video>
Apr 12 03:48:16 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:48:16 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:48:16 user nova-compute[72081]:     </rng>
Apr 12 03:48:16 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:48:16 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:48:16 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:48:16 user nova-compute[72081]:     </memballoon>
Apr 12 03:48:16 user nova-compute[72081]:   </devices>
Apr 12 03:48:16 user nova-compute[72081]: </domain>
Apr 12 03:48:16 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:48:16 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:48:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-682928462',display_name='tempest-TestVolumeBootPattern-server-682928462',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-682928462',id=37,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBASnyncVOhHKPzClYx3zwpDcJrieJkBwHOEHWlg2NFTJJdBa1/TdotUOLxC4yYnCsrHxIQy7r8qgZsehiquLQNSYVTnDsRU6DmQUCbMUYEphptEyPo7yf2/NkfI88nssBw==',key_name='tempest-TestVolumeBootPattern-54080024',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='003688353e2f4b36831c574ebaf65173',ramdisk_id='',reservation_id='r-ioaiilyi',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-235043461',owner_user_name='tempest-TestVolumeBootPattern-235043461-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:48:11Z,user_data=None,user_id='b00b348933224558aad1cb12921f100f',uuid=6e8d7159-e7c4-4dbd-ab06-074b1d4e3873,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c06ddd73-9521-41d8-a907-7af844b2adac", "address": "fa:16:3e:89:fe:e4", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc06ddd73-95", "ovs_interfaceid": "c06ddd73-9521-41d8-a907-7af844b2adac", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:48:16 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converting VIF {"id": "c06ddd73-9521-41d8-a907-7af844b2adac", "address": "fa:16:3e:89:fe:e4", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc06ddd73-95", "ovs_interfaceid": "c06ddd73-9521-41d8-a907-7af844b2adac", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:48:16 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:89:fe:e4,bridge_name='br-int',has_traffic_filtering=True,id=c06ddd73-9521-41d8-a907-7af844b2adac,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc06ddd73-95') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:48:16 user nova-compute[72081]: DEBUG os_vif [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:89:fe:e4,bridge_name='br-int',has_traffic_filtering=True,id=c06ddd73-9521-41d8-a907-7af844b2adac,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc06ddd73-95') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:48:16 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:16 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:48:16 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:48:16 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:16 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapc06ddd73-95, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:48:16 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapc06ddd73-95, col_values=(('external_ids', {'iface-id': 'c06ddd73-9521-41d8-a907-7af844b2adac', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:89:fe:e4', 'vm-uuid': '6e8d7159-e7c4-4dbd-ab06-074b1d4e3873'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:48:16 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:16 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:48:16 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:16 user nova-compute[72081]: INFO os_vif [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:89:fe:e4,bridge_name='br-int',has_traffic_filtering=True,id=c06ddd73-9521-41d8-a907-7af844b2adac,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc06ddd73-95')
Apr 12 03:48:16 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:48:16 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] No VIF found with MAC fa:16:3e:89:fe:e4, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:48:17 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:48:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:17 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:48:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:18 user nova-compute[72081]: DEBUG nova.compute.manager [req-6de1938a-53cb-4729-8cf4-3aae79f70f8f req-1bd84cc2-448b-452c-a7fb-f5293c0b589b service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Received event network-vif-plugged-c06ddd73-9521-41d8-a907-7af844b2adac {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:48:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6de1938a-53cb-4729-8cf4-3aae79f70f8f req-1bd84cc2-448b-452c-a7fb-f5293c0b589b service nova] Acquiring lock "6e8d7159-e7c4-4dbd-ab06-074b1d4e3873-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6de1938a-53cb-4729-8cf4-3aae79f70f8f req-1bd84cc2-448b-452c-a7fb-f5293c0b589b service nova] Lock "6e8d7159-e7c4-4dbd-ab06-074b1d4e3873-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6de1938a-53cb-4729-8cf4-3aae79f70f8f req-1bd84cc2-448b-452c-a7fb-f5293c0b589b service nova] Lock "6e8d7159-e7c4-4dbd-ab06-074b1d4e3873-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:18 user nova-compute[72081]: DEBUG nova.compute.manager [req-6de1938a-53cb-4729-8cf4-3aae79f70f8f req-1bd84cc2-448b-452c-a7fb-f5293c0b589b service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] No waiting events found dispatching network-vif-plugged-c06ddd73-9521-41d8-a907-7af844b2adac {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:48:18 user nova-compute[72081]: WARNING nova.compute.manager [req-6de1938a-53cb-4729-8cf4-3aae79f70f8f req-1bd84cc2-448b-452c-a7fb-f5293c0b589b service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Received unexpected event network-vif-plugged-c06ddd73-9521-41d8-a907-7af844b2adac for instance with vm_state building and task_state spawning.
Apr 12 03:48:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:18 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:48:18 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:48:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:48:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:19 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271299.8527417, 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:48:19 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] VM Resumed (Lifecycle Event)
Apr 12 03:48:19 user nova-compute[72081]: DEBUG nova.compute.manager [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:48:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:48:19 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:48:19 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:48:19 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8272MB free_disk=26.019519805908203GB free_vcpus=6 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:48:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:19 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:48:19 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Instance spawned successfully.
Apr 12 03:48:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:48:19 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:48:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:48:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:48:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:48:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:48:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:48:19 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:48:19 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:48:19 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271299.8535125, 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:48:19 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] VM Started (Lifecycle Event)
Apr 12 03:48:20 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:48:20 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:48:20 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 2d0904c3-74e0-47bb-a832-758a7035b2f5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:48:20 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance bcd267d2-5b2c-4ed2-b634-0210fae94dfd actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:48:20 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 606425bb-8968-41f9-b3a5-80b6fd188a67 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:48:20 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 7c2344ea-3ed4-40f5-b318-6fa8588b1fed actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:48:20 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 12351dbd-0f2e-40a6-a10a-f40f5b6250ef actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:48:20 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:48:20 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:48:20 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 7 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:48:20 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=1408MB phys_disk=40GB used_disk=4GB total_vcpus=12 used_vcpus=7 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:48:20 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:48:20 user nova-compute[72081]: INFO nova.compute.manager [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Took 5.02 seconds to spawn the instance on the hypervisor.
Apr 12 03:48:20 user nova-compute[72081]: DEBUG nova.compute.manager [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:48:20 user nova-compute[72081]: DEBUG nova.compute.manager [req-1ae3ac70-ea7f-499a-948d-58f7b299bb8c req-4359036e-93b1-44eb-bbdf-50a0dc7cde06 service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Received event network-vif-plugged-c06ddd73-9521-41d8-a907-7af844b2adac {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:48:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1ae3ac70-ea7f-499a-948d-58f7b299bb8c req-4359036e-93b1-44eb-bbdf-50a0dc7cde06 service nova] Acquiring lock "6e8d7159-e7c4-4dbd-ab06-074b1d4e3873-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1ae3ac70-ea7f-499a-948d-58f7b299bb8c req-4359036e-93b1-44eb-bbdf-50a0dc7cde06 service nova] Lock "6e8d7159-e7c4-4dbd-ab06-074b1d4e3873-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1ae3ac70-ea7f-499a-948d-58f7b299bb8c req-4359036e-93b1-44eb-bbdf-50a0dc7cde06 service nova] Lock "6e8d7159-e7c4-4dbd-ab06-074b1d4e3873-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:20 user nova-compute[72081]: DEBUG nova.compute.manager [req-1ae3ac70-ea7f-499a-948d-58f7b299bb8c req-4359036e-93b1-44eb-bbdf-50a0dc7cde06 service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] No waiting events found dispatching network-vif-plugged-c06ddd73-9521-41d8-a907-7af844b2adac {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:48:20 user nova-compute[72081]: WARNING nova.compute.manager [req-1ae3ac70-ea7f-499a-948d-58f7b299bb8c req-4359036e-93b1-44eb-bbdf-50a0dc7cde06 service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Received unexpected event network-vif-plugged-c06ddd73-9521-41d8-a907-7af844b2adac for instance with vm_state building and task_state spawning.
Apr 12 03:48:20 user nova-compute[72081]: INFO nova.compute.manager [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Took 10.10 seconds to build instance.
Apr 12 03:48:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-672a0050-4d11-49f6-a16b-e8d0129781d1 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "6e8d7159-e7c4-4dbd-ab06-074b1d4e3873" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 10.195s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:20 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:48:20 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:48:20 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:48:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.374s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:20 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681271285.6075346, 3d4407f4-e078-4777-a110-cb3f922be2e8 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:48:20 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] VM Stopped (Lifecycle Event)
Apr 12 03:48:20 user nova-compute[72081]: DEBUG nova.compute.manager [None req-2c612f6f-225f-4783-b43d-04003048b101 None None] [instance: 3d4407f4-e078-4777-a110-cb3f922be2e8] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:48:21 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:48:21 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:48:21 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:48:21 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:48:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:21 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:48:21 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:48:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-606425bb-8968-41f9-b3a5-80b6fd188a67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:48:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-606425bb-8968-41f9-b3a5-80b6fd188a67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:48:21 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 03:48:22 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Updating instance_info_cache with network_info: [{"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.205", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:48:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-606425bb-8968-41f9-b3a5-80b6fd188a67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:48:22 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 03:48:23 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:48:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:24 user nova-compute[72081]: DEBUG nova.compute.manager [None req-e4f10acb-86dc-47d5-909f-8fe0b8fa2753 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:48:25 user nova-compute[72081]: INFO nova.compute.manager [None req-e4f10acb-86dc-47d5-909f-8fe0b8fa2753 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] instance snapshotting
Apr 12 03:48:25 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-e4f10acb-86dc-47d5-909f-8fe0b8fa2753 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Beginning live snapshot process
Apr 12 03:48:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-e4f10acb-86dc-47d5-909f-8fe0b8fa2753 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json -f qcow2 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-e4f10acb-86dc-47d5-909f-8fe0b8fa2753 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json -f qcow2" returned: 0 in 0.152s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-e4f10acb-86dc-47d5-909f-8fe0b8fa2753 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json -f qcow2 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-e4f10acb-86dc-47d5-909f-8fe0b8fa2753 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json -f qcow2" returned: 0 in 0.147s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-e4f10acb-86dc-47d5-909f-8fe0b8fa2753 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-e4f10acb-86dc-47d5-909f-8fe0b8fa2753 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.131s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-e4f10acb-86dc-47d5-909f-8fe0b8fa2753 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmprixd48m2/d36493cffbf640ae8eb292cc539c4c90.delta 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-e4f10acb-86dc-47d5-909f-8fe0b8fa2753 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmprixd48m2/d36493cffbf640ae8eb292cc539c4c90.delta 1073741824" returned: 0 in 0.051s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:25 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-e4f10acb-86dc-47d5-909f-8fe0b8fa2753 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Quiescing instance not available: QEMU guest agent is not enabled.
Apr 12 03:48:26 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-e4f10acb-86dc-47d5-909f-8fe0b8fa2753 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] COPY block job progress, current cursor: 0 final cursor: 43843584 {{(pid=72081) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 12 03:48:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:26 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-e4f10acb-86dc-47d5-909f-8fe0b8fa2753 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] COPY block job progress, current cursor: 43843584 final cursor: 43843584 {{(pid=72081) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 12 03:48:26 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-e4f10acb-86dc-47d5-909f-8fe0b8fa2753 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Skipping quiescing instance: QEMU guest agent is not enabled.
Apr 12 03:48:27 user nova-compute[72081]: DEBUG nova.privsep.utils [None req-e4f10acb-86dc-47d5-909f-8fe0b8fa2753 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72081) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 12 03:48:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-e4f10acb-86dc-47d5-909f-8fe0b8fa2753 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmprixd48m2/d36493cffbf640ae8eb292cc539c4c90.delta /opt/stack/data/nova/instances/snapshots/tmprixd48m2/d36493cffbf640ae8eb292cc539c4c90 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-e4f10acb-86dc-47d5-909f-8fe0b8fa2753 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmprixd48m2/d36493cffbf640ae8eb292cc539c4c90.delta /opt/stack/data/nova/instances/snapshots/tmprixd48m2/d36493cffbf640ae8eb292cc539c4c90" returned: 0 in 0.392s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:27 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-e4f10acb-86dc-47d5-909f-8fe0b8fa2753 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Snapshot extracted, beginning image upload
Apr 12 03:48:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:29 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-e4f10acb-86dc-47d5-909f-8fe0b8fa2753 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Snapshot image upload complete
Apr 12 03:48:29 user nova-compute[72081]: INFO nova.compute.manager [None req-e4f10acb-86dc-47d5-909f-8fe0b8fa2753 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Took 4.54 seconds to snapshot the instance on the hypervisor.
Apr 12 03:48:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "a6d7d43b-ac65-4ebf-9645-b17e61397076" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "a6d7d43b-ac65-4ebf-9645-b17e61397076" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:48:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:33 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:48:33 user nova-compute[72081]: INFO nova.compute.claims [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Claim successful on node user
Apr 12 03:48:34 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:48:34 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:48:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.370s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:34 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:48:34 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:48:34 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:48:34 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:48:34 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:48:34 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:48:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:48:34 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Creating image(s)
Apr 12 03:48:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "/opt/stack/data/nova/instances/a6d7d43b-ac65-4ebf-9645-b17e61397076/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "/opt/stack/data/nova/instances/a6d7d43b-ac65-4ebf-9645-b17e61397076/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "/opt/stack/data/nova/instances/a6d7d43b-ac65-4ebf-9645-b17e61397076/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.007s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "52bac5158985566ef9844ce516d5fe47fe83736c" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "52bac5158985566ef9844ce516d5fe47fe83736c" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:34 user nova-compute[72081]: DEBUG nova.policy [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '88c496d8e488434e833e67df25def576', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c426e52304c14ed9b0c1aed69691bc3d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:48:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/52bac5158985566ef9844ce516d5fe47fe83736c.part --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/52bac5158985566ef9844ce516d5fe47fe83736c.part --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:35 user nova-compute[72081]: DEBUG nova.virt.images [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] e36d7fe2-0f9a-43e7-9565-714b8a727ea5 was qcow2, converting to raw {{(pid=72081) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 12 03:48:35 user nova-compute[72081]: DEBUG nova.privsep.utils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72081) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 12 03:48:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/52bac5158985566ef9844ce516d5fe47fe83736c.part /opt/stack/data/nova/instances/_base/52bac5158985566ef9844ce516d5fe47fe83736c.converted {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/52bac5158985566ef9844ce516d5fe47fe83736c.part /opt/stack/data/nova/instances/_base/52bac5158985566ef9844ce516d5fe47fe83736c.converted" returned: 0 in 0.642s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/52bac5158985566ef9844ce516d5fe47fe83736c.converted --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Successfully created port: 7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/52bac5158985566ef9844ce516d5fe47fe83736c.converted --force-share --output=json" returned: 0 in 0.150s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "52bac5158985566ef9844ce516d5fe47fe83736c" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.855s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/52bac5158985566ef9844ce516d5fe47fe83736c --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/52bac5158985566ef9844ce516d5fe47fe83736c --force-share --output=json" returned: 0 in 0.133s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "52bac5158985566ef9844ce516d5fe47fe83736c" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "52bac5158985566ef9844ce516d5fe47fe83736c" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/52bac5158985566ef9844ce516d5fe47fe83736c --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/52bac5158985566ef9844ce516d5fe47fe83736c --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/52bac5158985566ef9844ce516d5fe47fe83736c,backing_fmt=raw /opt/stack/data/nova/instances/a6d7d43b-ac65-4ebf-9645-b17e61397076/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/52bac5158985566ef9844ce516d5fe47fe83736c,backing_fmt=raw /opt/stack/data/nova/instances/a6d7d43b-ac65-4ebf-9645-b17e61397076/disk 1073741824" returned: 0 in 0.046s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "52bac5158985566ef9844ce516d5fe47fe83736c" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.188s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/52bac5158985566ef9844ce516d5fe47fe83736c --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/52bac5158985566ef9844ce516d5fe47fe83736c --force-share --output=json" returned: 0 in 0.130s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG nova.objects.instance [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lazy-loading 'migration_context' on Instance uuid a6d7d43b-ac65-4ebf-9645-b17e61397076 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Ensure instance console log exists: /opt/stack/data/nova/instances/a6d7d43b-ac65-4ebf-9645-b17e61397076/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Successfully updated port: 7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "refresh_cache-a6d7d43b-ac65-4ebf-9645-b17e61397076" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquired lock "refresh_cache-a6d7d43b-ac65-4ebf-9645-b17e61397076" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:48:36 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-09ec884f-7808-4a16-8011-ab2f4fe195ac req-9651587c-1d4e-4def-bf8e-aa68f8ae6b28 service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Received event network-changed-7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-09ec884f-7808-4a16-8011-ab2f4fe195ac req-9651587c-1d4e-4def-bf8e-aa68f8ae6b28 service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Refreshing instance network info cache due to event network-changed-7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-09ec884f-7808-4a16-8011-ab2f4fe195ac req-9651587c-1d4e-4def-bf8e-aa68f8ae6b28 service nova] Acquiring lock "refresh_cache-a6d7d43b-ac65-4ebf-9645-b17e61397076" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Updating instance_info_cache with network_info: [{"id": "7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913", "address": "fa:16:3e:78:3c:6d", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f861491-1d", "ovs_interfaceid": "7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Releasing lock "refresh_cache-a6d7d43b-ac65-4ebf-9645-b17e61397076" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Instance network_info: |[{"id": "7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913", "address": "fa:16:3e:78:3c:6d", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f861491-1d", "ovs_interfaceid": "7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-09ec884f-7808-4a16-8011-ab2f4fe195ac req-9651587c-1d4e-4def-bf8e-aa68f8ae6b28 service nova] Acquired lock "refresh_cache-a6d7d43b-ac65-4ebf-9645-b17e61397076" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.network.neutron [req-09ec884f-7808-4a16-8011-ab2f4fe195ac req-9651587c-1d4e-4def-bf8e-aa68f8ae6b28 service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Refreshing network info cache for port 7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Start _get_guest_xml network_info=[{"id": "7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913", "address": "fa:16:3e:78:3c:6d", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f861491-1d", "ovs_interfaceid": "7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='ae76233b530251021d8928dfd1709b40',container_format='bare',created_at=2023-04-12T03:48:25Z,direct_url=<?>,disk_format='qcow2',id=e36d7fe2-0f9a-43e7-9565-714b8a727ea5,min_disk=1,min_ram=0,name='tempest-TestStampPatternsnapshot-1998485426',owner='c426e52304c14ed9b0c1aed69691bc3d',properties=ImageMetaProps,protected=<?>,size=38666240,status='active',tags=<?>,updated_at=2023-04-12T03:48:29Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e36d7fe2-0f9a-43e7-9565-714b8a727ea5'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:48:37 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:48:37 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='ae76233b530251021d8928dfd1709b40',container_format='bare',created_at=2023-04-12T03:48:25Z,direct_url=<?>,disk_format='qcow2',id=e36d7fe2-0f9a-43e7-9565-714b8a727ea5,min_disk=1,min_ram=0,name='tempest-TestStampPatternsnapshot-1998485426',owner='c426e52304c14ed9b0c1aed69691bc3d',properties=ImageMetaProps,protected=<?>,size=38666240,status='active',tags=<?>,updated_at=2023-04-12T03:48:29Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:48:33Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestStampPattern-server-2010836582',display_name='tempest-TestStampPattern-server-2010836582',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-teststamppattern-server-2010836582',id=38,image_ref='e36d7fe2-0f9a-43e7-9565-714b8a727ea5',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJxviRdZviloE/Y8vy1de4Nl8WuqpnNhf4wG8IpFYt8u0P1qgKwJzMzVj8wiWWgzOhRDflg4OM3oRDBzEH9LU9bTdJCvVUrt7G9IZqwugvdWPxD2R1lvN0sxiTneX8pngQ==',key_name='tempest-TestStampPattern-474439147',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='c426e52304c14ed9b0c1aed69691bc3d',ramdisk_id='',reservation_id='r-0aywectj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_boot_roles='reader,member',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_image_location='snapshot',image_image_state='available',image_image_type='snapshot',image_instance_uuid='7c2344ea-3ed4-40f5-b318-6fa8588b1fed',image_min_disk='1',image_min_ram='0',image_owner_id='c426e52304c14ed9b0c1aed69691bc3d',image_owner_project_name='tempest-TestStampPattern-707517070',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestStampPattern-707517070-project-member',image_user_id='88c496d8e488434e833e67df25def576',network_allocated='True',owner_project_name='tempest-TestStampPattern-707517070',owner_user_name='tempest-TestStampPattern-707517070-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:48:34Z,user_data=None,user_id='88c496d8e488434e833e67df25def576',uuid=a6d7d43b-ac65-4ebf-9645-b17e61397076,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913", "address": "fa:16:3e:78:3c:6d", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f861491-1d", "ovs_interfaceid": "7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Converting VIF {"id": "7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913", "address": "fa:16:3e:78:3c:6d", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f861491-1d", "ovs_interfaceid": "7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:78:3c:6d,bridge_name='br-int',has_traffic_filtering=True,id=7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913,network=Network(8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7f861491-1d') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.objects.instance [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lazy-loading 'pci_devices' on Instance uuid a6d7d43b-ac65-4ebf-9645-b17e61397076 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:48:37 user nova-compute[72081]:   <uuid>a6d7d43b-ac65-4ebf-9645-b17e61397076</uuid>
Apr 12 03:48:37 user nova-compute[72081]:   <name>instance-00000026</name>
Apr 12 03:48:37 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:48:37 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:48:37 user nova-compute[72081]:   <metadata>
Apr 12 03:48:37 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:48:37 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:48:37 user nova-compute[72081]:       <nova:name>tempest-TestStampPattern-server-2010836582</nova:name>
Apr 12 03:48:37 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:48:37</nova:creationTime>
Apr 12 03:48:37 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:48:37 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:48:37 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:48:37 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:48:37 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:48:37 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:48:37 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:48:37 user nova-compute[72081]:       <nova:owner>
Apr 12 03:48:37 user nova-compute[72081]:         <nova:user uuid="88c496d8e488434e833e67df25def576">tempest-TestStampPattern-707517070-project-member</nova:user>
Apr 12 03:48:37 user nova-compute[72081]:         <nova:project uuid="c426e52304c14ed9b0c1aed69691bc3d">tempest-TestStampPattern-707517070</nova:project>
Apr 12 03:48:37 user nova-compute[72081]:       </nova:owner>
Apr 12 03:48:37 user nova-compute[72081]:       <nova:root type="image" uuid="e36d7fe2-0f9a-43e7-9565-714b8a727ea5"/>
Apr 12 03:48:37 user nova-compute[72081]:       <nova:ports>
Apr 12 03:48:37 user nova-compute[72081]:         <nova:port uuid="7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913">
Apr 12 03:48:37 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.7" ipVersion="4"/>
Apr 12 03:48:37 user nova-compute[72081]:         </nova:port>
Apr 12 03:48:37 user nova-compute[72081]:       </nova:ports>
Apr 12 03:48:37 user nova-compute[72081]:     </nova:instance>
Apr 12 03:48:37 user nova-compute[72081]:   </metadata>
Apr 12 03:48:37 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:48:37 user nova-compute[72081]:     <system>
Apr 12 03:48:37 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:48:37 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:48:37 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:48:37 user nova-compute[72081]:       <entry name="serial">a6d7d43b-ac65-4ebf-9645-b17e61397076</entry>
Apr 12 03:48:37 user nova-compute[72081]:       <entry name="uuid">a6d7d43b-ac65-4ebf-9645-b17e61397076</entry>
Apr 12 03:48:37 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:48:37 user nova-compute[72081]:     </system>
Apr 12 03:48:37 user nova-compute[72081]:   </sysinfo>
Apr 12 03:48:37 user nova-compute[72081]:   <os>
Apr 12 03:48:37 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:48:37 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:48:37 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:48:37 user nova-compute[72081]:   </os>
Apr 12 03:48:37 user nova-compute[72081]:   <features>
Apr 12 03:48:37 user nova-compute[72081]:     <acpi/>
Apr 12 03:48:37 user nova-compute[72081]:     <apic/>
Apr 12 03:48:37 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:48:37 user nova-compute[72081]:   </features>
Apr 12 03:48:37 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:48:37 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:48:37 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:48:37 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:48:37 user nova-compute[72081]:   </clock>
Apr 12 03:48:37 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:48:37 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:48:37 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:48:37 user nova-compute[72081]:   </cpu>
Apr 12 03:48:37 user nova-compute[72081]:   <devices>
Apr 12 03:48:37 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:48:37 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:48:37 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/a6d7d43b-ac65-4ebf-9645-b17e61397076/disk"/>
Apr 12 03:48:37 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:48:37 user nova-compute[72081]:     </disk>
Apr 12 03:48:37 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:48:37 user nova-compute[72081]:       <mac address="fa:16:3e:78:3c:6d"/>
Apr 12 03:48:37 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:48:37 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:48:37 user nova-compute[72081]:       <target dev="tap7f861491-1d"/>
Apr 12 03:48:37 user nova-compute[72081]:     </interface>
Apr 12 03:48:37 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:48:37 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/a6d7d43b-ac65-4ebf-9645-b17e61397076/console.log" append="off"/>
Apr 12 03:48:37 user nova-compute[72081]:     </serial>
Apr 12 03:48:37 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:48:37 user nova-compute[72081]:     <video>
Apr 12 03:48:37 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:48:37 user nova-compute[72081]:     </video>
Apr 12 03:48:37 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:48:37 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:48:37 user nova-compute[72081]:     </rng>
Apr 12 03:48:37 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:48:37 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:48:37 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:48:37 user nova-compute[72081]:     </memballoon>
Apr 12 03:48:37 user nova-compute[72081]:   </devices>
Apr 12 03:48:37 user nova-compute[72081]: </domain>
Apr 12 03:48:37 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:48:33Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestStampPattern-server-2010836582',display_name='tempest-TestStampPattern-server-2010836582',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-teststamppattern-server-2010836582',id=38,image_ref='e36d7fe2-0f9a-43e7-9565-714b8a727ea5',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJxviRdZviloE/Y8vy1de4Nl8WuqpnNhf4wG8IpFYt8u0P1qgKwJzMzVj8wiWWgzOhRDflg4OM3oRDBzEH9LU9bTdJCvVUrt7G9IZqwugvdWPxD2R1lvN0sxiTneX8pngQ==',key_name='tempest-TestStampPattern-474439147',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='c426e52304c14ed9b0c1aed69691bc3d',ramdisk_id='',reservation_id='r-0aywectj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_boot_roles='reader,member',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_image_location='snapshot',image_image_state='available',image_image_type='snapshot',image_instance_uuid='7c2344ea-3ed4-40f5-b318-6fa8588b1fed',image_min_disk='1',image_min_ram='0',image_owner_id='c426e52304c14ed9b0c1aed69691bc3d',image_owner_project_name='tempest-TestStampPattern-707517070',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestStampPattern-707517070-project-member',image_user_id='88c496d8e488434e833e67df25def576',network_allocated='True',owner_project_name='tempest-TestStampPattern-707517070',owner_user_name='tempest-TestStampPattern-707517070-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:48:34Z,user_data=None,user_id='88c496d8e488434e833e67df25def576',uuid=a6d7d43b-ac65-4ebf-9645-b17e61397076,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913", "address": "fa:16:3e:78:3c:6d", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f861491-1d", "ovs_interfaceid": "7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Converting VIF {"id": "7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913", "address": "fa:16:3e:78:3c:6d", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f861491-1d", "ovs_interfaceid": "7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:78:3c:6d,bridge_name='br-int',has_traffic_filtering=True,id=7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913,network=Network(8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7f861491-1d') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG os_vif [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:78:3c:6d,bridge_name='br-int',has_traffic_filtering=True,id=7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913,network=Network(8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7f861491-1d') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap7f861491-1d, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap7f861491-1d, col_values=(('external_ids', {'iface-id': '7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:78:3c:6d', 'vm-uuid': 'a6d7d43b-ac65-4ebf-9645-b17e61397076'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:37 user nova-compute[72081]: INFO os_vif [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:78:3c:6d,bridge_name='br-int',has_traffic_filtering=True,id=7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913,network=Network(8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7f861491-1d')
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] No VIF found with MAC fa:16:3e:78:3c:6d, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.network.neutron [req-09ec884f-7808-4a16-8011-ab2f4fe195ac req-9651587c-1d4e-4def-bf8e-aa68f8ae6b28 service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Updated VIF entry in instance network info cache for port 7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG nova.network.neutron [req-09ec884f-7808-4a16-8011-ab2f4fe195ac req-9651587c-1d4e-4def-bf8e-aa68f8ae6b28 service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Updating instance_info_cache with network_info: [{"id": "7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913", "address": "fa:16:3e:78:3c:6d", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f861491-1d", "ovs_interfaceid": "7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:48:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-09ec884f-7808-4a16-8011-ab2f4fe195ac req-9651587c-1d4e-4def-bf8e-aa68f8ae6b28 service nova] Releasing lock "refresh_cache-a6d7d43b-ac65-4ebf-9645-b17e61397076" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:48:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:39 user nova-compute[72081]: DEBUG nova.compute.manager [req-b3fba22c-abe6-46ed-8ca4-3e222cf1ec31 req-a412431a-b096-4f9e-a46f-7d0811d70bb7 service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Received event network-vif-plugged-7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:48:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b3fba22c-abe6-46ed-8ca4-3e222cf1ec31 req-a412431a-b096-4f9e-a46f-7d0811d70bb7 service nova] Acquiring lock "a6d7d43b-ac65-4ebf-9645-b17e61397076-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b3fba22c-abe6-46ed-8ca4-3e222cf1ec31 req-a412431a-b096-4f9e-a46f-7d0811d70bb7 service nova] Lock "a6d7d43b-ac65-4ebf-9645-b17e61397076-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b3fba22c-abe6-46ed-8ca4-3e222cf1ec31 req-a412431a-b096-4f9e-a46f-7d0811d70bb7 service nova] Lock "a6d7d43b-ac65-4ebf-9645-b17e61397076-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:39 user nova-compute[72081]: DEBUG nova.compute.manager [req-b3fba22c-abe6-46ed-8ca4-3e222cf1ec31 req-a412431a-b096-4f9e-a46f-7d0811d70bb7 service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] No waiting events found dispatching network-vif-plugged-7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:48:39 user nova-compute[72081]: WARNING nova.compute.manager [req-b3fba22c-abe6-46ed-8ca4-3e222cf1ec31 req-a412431a-b096-4f9e-a46f-7d0811d70bb7 service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Received unexpected event network-vif-plugged-7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913 for instance with vm_state building and task_state spawning.
Apr 12 03:48:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:40 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271320.893821, a6d7d43b-ac65-4ebf-9645-b17e61397076 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:48:40 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] VM Resumed (Lifecycle Event)
Apr 12 03:48:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:48:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:48:40 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Instance spawned successfully.
Apr 12 03:48:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Attempting to register defaults for the following image properties: ['hw_input_bus', 'hw_pointer_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:48:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:48:40 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:48:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:48:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:48:40 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:48:40 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271320.8952982, a6d7d43b-ac65-4ebf-9645-b17e61397076 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:48:40 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] VM Started (Lifecycle Event)
Apr 12 03:48:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:48:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:48:40 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:48:41 user nova-compute[72081]: INFO nova.compute.manager [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Took 6.53 seconds to spawn the instance on the hypervisor.
Apr 12 03:48:41 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:48:41 user nova-compute[72081]: INFO nova.compute.manager [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Took 7.23 seconds to build instance.
Apr 12 03:48:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b08c399d-6c6d-4b7f-a109-e6604ee1e877 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "a6d7d43b-ac65-4ebf-9645-b17e61397076" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.325s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:41 user nova-compute[72081]: DEBUG nova.compute.manager [req-a35a2ff7-89c4-42c2-8737-a5285c16ccbd req-f14135a3-a300-4ac5-a2ef-763b71914dde service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Received event network-vif-plugged-7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:48:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a35a2ff7-89c4-42c2-8737-a5285c16ccbd req-f14135a3-a300-4ac5-a2ef-763b71914dde service nova] Acquiring lock "a6d7d43b-ac65-4ebf-9645-b17e61397076-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a35a2ff7-89c4-42c2-8737-a5285c16ccbd req-f14135a3-a300-4ac5-a2ef-763b71914dde service nova] Lock "a6d7d43b-ac65-4ebf-9645-b17e61397076-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a35a2ff7-89c4-42c2-8737-a5285c16ccbd req-f14135a3-a300-4ac5-a2ef-763b71914dde service nova] Lock "a6d7d43b-ac65-4ebf-9645-b17e61397076-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:41 user nova-compute[72081]: DEBUG nova.compute.manager [req-a35a2ff7-89c4-42c2-8737-a5285c16ccbd req-f14135a3-a300-4ac5-a2ef-763b71914dde service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] No waiting events found dispatching network-vif-plugged-7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:48:41 user nova-compute[72081]: WARNING nova.compute.manager [req-a35a2ff7-89c4-42c2-8737-a5285c16ccbd req-f14135a3-a300-4ac5-a2ef-763b71914dde service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Received unexpected event network-vif-plugged-7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913 for instance with vm_state active and task_state None.
Apr 12 03:48:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "93a2f40a-1eae-4986-b418-ddbf45723196" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "93a2f40a-1eae-4986-b418-ddbf45723196" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:51 user nova-compute[72081]: DEBUG nova.compute.manager [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:48:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:51 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:48:51 user nova-compute[72081]: INFO nova.compute.claims [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Claim successful on node user
Apr 12 03:48:52 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:48:52 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:48:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.402s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:52 user nova-compute[72081]: DEBUG nova.compute.manager [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:48:52 user nova-compute[72081]: DEBUG nova.compute.manager [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:48:52 user nova-compute[72081]: DEBUG nova.network.neutron [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:48:52 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:48:52 user nova-compute[72081]: DEBUG nova.compute.manager [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:48:52 user nova-compute[72081]: DEBUG nova.policy [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd52637cca2114e5e99e688ee371c835a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd13546149a4e44e1b782b99e9eb452e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:48:52 user nova-compute[72081]: DEBUG nova.compute.manager [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:48:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:48:52 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Creating image(s)
Apr 12 03:48:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "/opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "/opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "/opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.006s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.131s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.149s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196/disk 1073741824" returned: 0 in 0.054s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.209s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.148s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:52 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Checking if we can resize image /opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:48:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196/disk --force-share --output=json" returned: 0 in 0.163s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:53 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Cannot resize image /opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:48:53 user nova-compute[72081]: DEBUG nova.objects.instance [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lazy-loading 'migration_context' on Instance uuid 93a2f40a-1eae-4986-b418-ddbf45723196 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:48:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:48:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Ensure instance console log exists: /opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:48:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:53 user nova-compute[72081]: DEBUG nova.network.neutron [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Successfully created port: 7afd2456-5a12-4283-8760-46d4c7fe8ad1 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:48:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:53 user nova-compute[72081]: DEBUG nova.network.neutron [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Successfully updated port: 7afd2456-5a12-4283-8760-46d4c7fe8ad1 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:48:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "refresh_cache-93a2f40a-1eae-4986-b418-ddbf45723196" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:48:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquired lock "refresh_cache-93a2f40a-1eae-4986-b418-ddbf45723196" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:48:53 user nova-compute[72081]: DEBUG nova.network.neutron [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:48:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-68054a51-b5c9-4126-a8a7-3f4d69651f95 req-4b13aab8-7682-491e-901e-fbe73d78ea75 service nova] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Received event network-changed-7afd2456-5a12-4283-8760-46d4c7fe8ad1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:48:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-68054a51-b5c9-4126-a8a7-3f4d69651f95 req-4b13aab8-7682-491e-901e-fbe73d78ea75 service nova] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Refreshing instance network info cache due to event network-changed-7afd2456-5a12-4283-8760-46d4c7fe8ad1. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:48:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-68054a51-b5c9-4126-a8a7-3f4d69651f95 req-4b13aab8-7682-491e-901e-fbe73d78ea75 service nova] Acquiring lock "refresh_cache-93a2f40a-1eae-4986-b418-ddbf45723196" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:48:53 user nova-compute[72081]: DEBUG nova.network.neutron [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.network.neutron [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Updating instance_info_cache with network_info: [{"id": "7afd2456-5a12-4283-8760-46d4c7fe8ad1", "address": "fa:16:3e:ba:1d:ce", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7afd2456-5a", "ovs_interfaceid": "7afd2456-5a12-4283-8760-46d4c7fe8ad1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Releasing lock "refresh_cache-93a2f40a-1eae-4986-b418-ddbf45723196" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.compute.manager [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Instance network_info: |[{"id": "7afd2456-5a12-4283-8760-46d4c7fe8ad1", "address": "fa:16:3e:ba:1d:ce", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7afd2456-5a", "ovs_interfaceid": "7afd2456-5a12-4283-8760-46d4c7fe8ad1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-68054a51-b5c9-4126-a8a7-3f4d69651f95 req-4b13aab8-7682-491e-901e-fbe73d78ea75 service nova] Acquired lock "refresh_cache-93a2f40a-1eae-4986-b418-ddbf45723196" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.network.neutron [req-68054a51-b5c9-4126-a8a7-3f4d69651f95 req-4b13aab8-7682-491e-901e-fbe73d78ea75 service nova] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Refreshing network info cache for port 7afd2456-5a12-4283-8760-46d4c7fe8ad1 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Start _get_guest_xml network_info=[{"id": "7afd2456-5a12-4283-8760-46d4c7fe8ad1", "address": "fa:16:3e:ba:1d:ce", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7afd2456-5a", "ovs_interfaceid": "7afd2456-5a12-4283-8760-46d4c7fe8ad1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:48:54 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:48:54 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:48:51Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-127420323',display_name='tempest-AttachVolumeNegativeTest-server-127420323',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-127420323',id=39,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFMWaRzq0i18wkyi3ji2eDJZY0v9caN/UqoTYCms3zb7sFYi3nVI2quTKtNRnFhKWj2bjy1/Mkv/ZsXUW1QOTCPWsEbRbmvA2ipXmyTgXPzbmkqrBL4q39n/p2cfT7w3RA==',key_name='tempest-keypair-422441298',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='d13546149a4e44e1b782b99e9eb452e8',ramdisk_id='',reservation_id='r-om37s4hv',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-494105310',owner_user_name='tempest-AttachVolumeNegativeTest-494105310-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:48:52Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d52637cca2114e5e99e688ee371c835a',uuid=93a2f40a-1eae-4986-b418-ddbf45723196,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7afd2456-5a12-4283-8760-46d4c7fe8ad1", "address": "fa:16:3e:ba:1d:ce", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7afd2456-5a", "ovs_interfaceid": "7afd2456-5a12-4283-8760-46d4c7fe8ad1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Converting VIF {"id": "7afd2456-5a12-4283-8760-46d4c7fe8ad1", "address": "fa:16:3e:ba:1d:ce", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7afd2456-5a", "ovs_interfaceid": "7afd2456-5a12-4283-8760-46d4c7fe8ad1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ba:1d:ce,bridge_name='br-int',has_traffic_filtering=True,id=7afd2456-5a12-4283-8760-46d4c7fe8ad1,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7afd2456-5a') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.objects.instance [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lazy-loading 'pci_devices' on Instance uuid 93a2f40a-1eae-4986-b418-ddbf45723196 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:48:54 user nova-compute[72081]:   <uuid>93a2f40a-1eae-4986-b418-ddbf45723196</uuid>
Apr 12 03:48:54 user nova-compute[72081]:   <name>instance-00000027</name>
Apr 12 03:48:54 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:48:54 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:48:54 user nova-compute[72081]:   <metadata>
Apr 12 03:48:54 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:48:54 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:48:54 user nova-compute[72081]:       <nova:name>tempest-AttachVolumeNegativeTest-server-127420323</nova:name>
Apr 12 03:48:54 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:48:54</nova:creationTime>
Apr 12 03:48:54 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:48:54 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:48:54 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:48:54 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:48:54 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:48:54 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:48:54 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:48:54 user nova-compute[72081]:       <nova:owner>
Apr 12 03:48:54 user nova-compute[72081]:         <nova:user uuid="d52637cca2114e5e99e688ee371c835a">tempest-AttachVolumeNegativeTest-494105310-project-member</nova:user>
Apr 12 03:48:54 user nova-compute[72081]:         <nova:project uuid="d13546149a4e44e1b782b99e9eb452e8">tempest-AttachVolumeNegativeTest-494105310</nova:project>
Apr 12 03:48:54 user nova-compute[72081]:       </nova:owner>
Apr 12 03:48:54 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:48:54 user nova-compute[72081]:       <nova:ports>
Apr 12 03:48:54 user nova-compute[72081]:         <nova:port uuid="7afd2456-5a12-4283-8760-46d4c7fe8ad1">
Apr 12 03:48:54 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.11" ipVersion="4"/>
Apr 12 03:48:54 user nova-compute[72081]:         </nova:port>
Apr 12 03:48:54 user nova-compute[72081]:       </nova:ports>
Apr 12 03:48:54 user nova-compute[72081]:     </nova:instance>
Apr 12 03:48:54 user nova-compute[72081]:   </metadata>
Apr 12 03:48:54 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:48:54 user nova-compute[72081]:     <system>
Apr 12 03:48:54 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:48:54 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:48:54 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:48:54 user nova-compute[72081]:       <entry name="serial">93a2f40a-1eae-4986-b418-ddbf45723196</entry>
Apr 12 03:48:54 user nova-compute[72081]:       <entry name="uuid">93a2f40a-1eae-4986-b418-ddbf45723196</entry>
Apr 12 03:48:54 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:48:54 user nova-compute[72081]:     </system>
Apr 12 03:48:54 user nova-compute[72081]:   </sysinfo>
Apr 12 03:48:54 user nova-compute[72081]:   <os>
Apr 12 03:48:54 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:48:54 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:48:54 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:48:54 user nova-compute[72081]:   </os>
Apr 12 03:48:54 user nova-compute[72081]:   <features>
Apr 12 03:48:54 user nova-compute[72081]:     <acpi/>
Apr 12 03:48:54 user nova-compute[72081]:     <apic/>
Apr 12 03:48:54 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:48:54 user nova-compute[72081]:   </features>
Apr 12 03:48:54 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:48:54 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:48:54 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:48:54 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:48:54 user nova-compute[72081]:   </clock>
Apr 12 03:48:54 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:48:54 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:48:54 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:48:54 user nova-compute[72081]:   </cpu>
Apr 12 03:48:54 user nova-compute[72081]:   <devices>
Apr 12 03:48:54 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:48:54 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:48:54 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196/disk"/>
Apr 12 03:48:54 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:48:54 user nova-compute[72081]:     </disk>
Apr 12 03:48:54 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:48:54 user nova-compute[72081]:       <mac address="fa:16:3e:ba:1d:ce"/>
Apr 12 03:48:54 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:48:54 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:48:54 user nova-compute[72081]:       <target dev="tap7afd2456-5a"/>
Apr 12 03:48:54 user nova-compute[72081]:     </interface>
Apr 12 03:48:54 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:48:54 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196/console.log" append="off"/>
Apr 12 03:48:54 user nova-compute[72081]:     </serial>
Apr 12 03:48:54 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:48:54 user nova-compute[72081]:     <video>
Apr 12 03:48:54 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:48:54 user nova-compute[72081]:     </video>
Apr 12 03:48:54 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:48:54 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:48:54 user nova-compute[72081]:     </rng>
Apr 12 03:48:54 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:48:54 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:48:54 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:48:54 user nova-compute[72081]:     </memballoon>
Apr 12 03:48:54 user nova-compute[72081]:   </devices>
Apr 12 03:48:54 user nova-compute[72081]: </domain>
Apr 12 03:48:54 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:48:51Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-127420323',display_name='tempest-AttachVolumeNegativeTest-server-127420323',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-127420323',id=39,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFMWaRzq0i18wkyi3ji2eDJZY0v9caN/UqoTYCms3zb7sFYi3nVI2quTKtNRnFhKWj2bjy1/Mkv/ZsXUW1QOTCPWsEbRbmvA2ipXmyTgXPzbmkqrBL4q39n/p2cfT7w3RA==',key_name='tempest-keypair-422441298',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='d13546149a4e44e1b782b99e9eb452e8',ramdisk_id='',reservation_id='r-om37s4hv',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-494105310',owner_user_name='tempest-AttachVolumeNegativeTest-494105310-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:48:52Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d52637cca2114e5e99e688ee371c835a',uuid=93a2f40a-1eae-4986-b418-ddbf45723196,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7afd2456-5a12-4283-8760-46d4c7fe8ad1", "address": "fa:16:3e:ba:1d:ce", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7afd2456-5a", "ovs_interfaceid": "7afd2456-5a12-4283-8760-46d4c7fe8ad1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Converting VIF {"id": "7afd2456-5a12-4283-8760-46d4c7fe8ad1", "address": "fa:16:3e:ba:1d:ce", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7afd2456-5a", "ovs_interfaceid": "7afd2456-5a12-4283-8760-46d4c7fe8ad1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ba:1d:ce,bridge_name='br-int',has_traffic_filtering=True,id=7afd2456-5a12-4283-8760-46d4c7fe8ad1,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7afd2456-5a') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG os_vif [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:ba:1d:ce,bridge_name='br-int',has_traffic_filtering=True,id=7afd2456-5a12-4283-8760-46d4c7fe8ad1,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7afd2456-5a') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap7afd2456-5a, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap7afd2456-5a, col_values=(('external_ids', {'iface-id': '7afd2456-5a12-4283-8760-46d4c7fe8ad1', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:ba:1d:ce', 'vm-uuid': '93a2f40a-1eae-4986-b418-ddbf45723196'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:54 user nova-compute[72081]: INFO os_vif [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:ba:1d:ce,bridge_name='br-int',has_traffic_filtering=True,id=7afd2456-5a12-4283-8760-46d4c7fe8ad1,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7afd2456-5a')
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] No VIF found with MAC fa:16:3e:ba:1d:ce, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.network.neutron [req-68054a51-b5c9-4126-a8a7-3f4d69651f95 req-4b13aab8-7682-491e-901e-fbe73d78ea75 service nova] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Updated VIF entry in instance network info cache for port 7afd2456-5a12-4283-8760-46d4c7fe8ad1. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG nova.network.neutron [req-68054a51-b5c9-4126-a8a7-3f4d69651f95 req-4b13aab8-7682-491e-901e-fbe73d78ea75 service nova] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Updating instance_info_cache with network_info: [{"id": "7afd2456-5a12-4283-8760-46d4c7fe8ad1", "address": "fa:16:3e:ba:1d:ce", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7afd2456-5a", "ovs_interfaceid": "7afd2456-5a12-4283-8760-46d4c7fe8ad1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:48:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-68054a51-b5c9-4126-a8a7-3f4d69651f95 req-4b13aab8-7682-491e-901e-fbe73d78ea75 service nova] Releasing lock "refresh_cache-93a2f40a-1eae-4986-b418-ddbf45723196" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:48:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:55 user nova-compute[72081]: INFO nova.compute.manager [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Terminating instance
Apr 12 03:48:55 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:48:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG nova.compute.manager [req-5912368b-1182-43f5-8d01-4a04e0735b66 req-3e7dd77e-551c-4537-989f-853da37366b2 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received event network-vif-unplugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5912368b-1182-43f5-8d01-4a04e0735b66 req-3e7dd77e-551c-4537-989f-853da37366b2 service nova] Acquiring lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5912368b-1182-43f5-8d01-4a04e0735b66 req-3e7dd77e-551c-4537-989f-853da37366b2 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5912368b-1182-43f5-8d01-4a04e0735b66 req-3e7dd77e-551c-4537-989f-853da37366b2 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG nova.compute.manager [req-5912368b-1182-43f5-8d01-4a04e0735b66 req-3e7dd77e-551c-4537-989f-853da37366b2 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] No waiting events found dispatching network-vif-unplugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG nova.compute.manager [req-5912368b-1182-43f5-8d01-4a04e0735b66 req-3e7dd77e-551c-4537-989f-853da37366b2 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received event network-vif-unplugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG nova.compute.manager [req-5912368b-1182-43f5-8d01-4a04e0735b66 req-3e7dd77e-551c-4537-989f-853da37366b2 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5912368b-1182-43f5-8d01-4a04e0735b66 req-3e7dd77e-551c-4537-989f-853da37366b2 service nova] Acquiring lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5912368b-1182-43f5-8d01-4a04e0735b66 req-3e7dd77e-551c-4537-989f-853da37366b2 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5912368b-1182-43f5-8d01-4a04e0735b66 req-3e7dd77e-551c-4537-989f-853da37366b2 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG nova.compute.manager [req-5912368b-1182-43f5-8d01-4a04e0735b66 req-3e7dd77e-551c-4537-989f-853da37366b2 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] No waiting events found dispatching network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:48:56 user nova-compute[72081]: WARNING nova.compute.manager [req-5912368b-1182-43f5-8d01-4a04e0735b66 req-3e7dd77e-551c-4537-989f-853da37366b2 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received unexpected event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 for instance with vm_state active and task_state deleting.
Apr 12 03:48:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG nova.compute.manager [req-fb64a3a4-0ef1-4cad-8fed-4d2cb86f664a req-56fcc0b1-cd85-4b83-84a9-d9159c639576 service nova] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Received event network-vif-plugged-7afd2456-5a12-4283-8760-46d4c7fe8ad1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-fb64a3a4-0ef1-4cad-8fed-4d2cb86f664a req-56fcc0b1-cd85-4b83-84a9-d9159c639576 service nova] Acquiring lock "93a2f40a-1eae-4986-b418-ddbf45723196-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-fb64a3a4-0ef1-4cad-8fed-4d2cb86f664a req-56fcc0b1-cd85-4b83-84a9-d9159c639576 service nova] Lock "93a2f40a-1eae-4986-b418-ddbf45723196-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-fb64a3a4-0ef1-4cad-8fed-4d2cb86f664a req-56fcc0b1-cd85-4b83-84a9-d9159c639576 service nova] Lock "93a2f40a-1eae-4986-b418-ddbf45723196-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG nova.compute.manager [req-fb64a3a4-0ef1-4cad-8fed-4d2cb86f664a req-56fcc0b1-cd85-4b83-84a9-d9159c639576 service nova] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] No waiting events found dispatching network-vif-plugged-7afd2456-5a12-4283-8760-46d4c7fe8ad1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:48:56 user nova-compute[72081]: WARNING nova.compute.manager [req-fb64a3a4-0ef1-4cad-8fed-4d2cb86f664a req-56fcc0b1-cd85-4b83-84a9-d9159c639576 service nova] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Received unexpected event network-vif-plugged-7afd2456-5a12-4283-8760-46d4c7fe8ad1 for instance with vm_state building and task_state spawning.
Apr 12 03:48:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:56 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Instance destroyed successfully.
Apr 12 03:48:56 user nova-compute[72081]: DEBUG nova.objects.instance [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lazy-loading 'resources' on Instance uuid bcd267d2-5b2c-4ed2-b634-0210fae94dfd {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:33:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-206381688',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-206381688',id=13,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:35:10Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='6e1fa352a8c144a1aada7ce0d8818429',ramdisk_id='',reservation_id='r-p09cbbxf',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-590302458',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:36:56Z,user_data=None,user_id='d99be569057f432484e4a60b46882bbd',uuid=bcd267d2-5b2c-4ed2-b634-0210fae94dfd,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "address": "fa:16:3e:9d:0c:c2", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa4385731-05", "ovs_interfaceid": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converting VIF {"id": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "address": "fa:16:3e:9d:0c:c2", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa4385731-05", "ovs_interfaceid": "a4385731-05f2-4eac-a5bb-9a987a3f5472", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:9d:0c:c2,bridge_name='br-int',has_traffic_filtering=True,id=a4385731-05f2-4eac-a5bb-9a987a3f5472,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa4385731-05') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG os_vif [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:9d:0c:c2,bridge_name='br-int',has_traffic_filtering=True,id=a4385731-05f2-4eac-a5bb-9a987a3f5472,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa4385731-05') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapa4385731-05, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:56 user nova-compute[72081]: INFO os_vif [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:9d:0c:c2,bridge_name='br-int',has_traffic_filtering=True,id=a4385731-05f2-4eac-a5bb-9a987a3f5472,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa4385731-05')
Apr 12 03:48:56 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FfmHBZWX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:48:56 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:48:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6c65d663-6def-4ef5-b63d-3677d832db4b]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:48:56 user nova-compute[72081]: No targets found.
Apr 12 03:48:56 user nova-compute[72081]: iSNS:
Apr 12 03:48:56 user nova-compute[72081]: No targets found.
Apr 12 03:48:56 user nova-compute[72081]: STATIC:
Apr 12 03:48:56 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76
Apr 12 03:48:56 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:48:56 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:48:56 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27
Apr 12 03:48:56 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:48:56 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:48:56 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2
Apr 12 03:48:56 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:48:56 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:48:56 user nova-compute[72081]: FIRMWARE:
Apr 12 03:48:56 user nova-compute[72081]: No targets found.
Apr 12 03:48:56 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:48:56 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:48:56 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8a67e6fb-69bf-4a05-874c-84d47a84861d]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a232dab3-27a2-4faa-8d51-078bd84fcf80]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:48:56 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:48:56 user nova-compute[72081]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 (non-flash)
Apr 12 03:48:56 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 (non-flash)
Apr 12 03:48:56 user nova-compute[72081]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:48:56 user nova-compute[72081]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 (non-flash)
Apr 12 03:48:56 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fb41fee60>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2'): ({'sdb'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Removing single pathed devices sdb {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ecd3df98-8940-4ac2-88a8-59495f8815a2]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2c978a6-a0c0-4c9e-9b53-3990ffdd5798]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Flushing IO for device /dev/sdb {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ab0028df-186b-488e-8f64-d200fcfda6ca]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.041s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c421ec48-86f0-4cf8-b4d4-d17cae140c9e]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] SCSI volumes sdb have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[67d205fa-b328-44fd-b28b-9b539cc55700]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --logout" returned: 0 in 0.058s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4bfdddec-7f91-4511-b9e3-1354232092ad]: (4, ('Logging out of session [sid: 14, target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2, portal: 172.16.0.220,3260]\nLogout of [sid: 14, target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 14, target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2, portal: 172.16.0.220,3260]
Apr 12 03:48:56 user nova-compute[72081]: Logout of [sid: 14, target: iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2, portal: 172.16.0.220,3260] successful.
Apr 12 03:48:56 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2d07a0d-a1ec-4f14-b1b0-51e4fae2e2e2 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d45e1585-937a-4bba-aa57-82c6c9baa333]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.313s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] <== disconnect_volume: return (315ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:48:56 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Deleting instance files /opt/stack/data/nova/instances/bcd267d2-5b2c-4ed2-b634-0210fae94dfd_del
Apr 12 03:48:56 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Deletion of /opt/stack/data/nova/instances/bcd267d2-5b2c-4ed2-b634-0210fae94dfd_del complete
Apr 12 03:48:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:56 user nova-compute[72081]: INFO nova.compute.manager [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Took 1.55 seconds to destroy the instance on the hypervisor.
Apr 12 03:48:56 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:48:56 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:48:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:57 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:48:57 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Took 0.57 seconds to deallocate network for instance.
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-2a5f211b-3c42-47a7-a7c7-b20d14f64977 req-69eb3e2d-23f5-45af-a3f2-fe74edcef187 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2a5f211b-3c42-47a7-a7c7-b20d14f64977 req-69eb3e2d-23f5-45af-a3f2-fe74edcef187 service nova] Acquiring lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2a5f211b-3c42-47a7-a7c7-b20d14f64977 req-69eb3e2d-23f5-45af-a3f2-fe74edcef187 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2a5f211b-3c42-47a7-a7c7-b20d14f64977 req-69eb3e2d-23f5-45af-a3f2-fe74edcef187 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-2a5f211b-3c42-47a7-a7c7-b20d14f64977 req-69eb3e2d-23f5-45af-a3f2-fe74edcef187 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] No waiting events found dispatching network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:48:58 user nova-compute[72081]: WARNING nova.compute.manager [req-2a5f211b-3c42-47a7-a7c7-b20d14f64977 req-69eb3e2d-23f5-45af-a3f2-fe74edcef187 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received unexpected event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 for instance with vm_state active and task_state deleting.
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-2a5f211b-3c42-47a7-a7c7-b20d14f64977 req-69eb3e2d-23f5-45af-a3f2-fe74edcef187 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2a5f211b-3c42-47a7-a7c7-b20d14f64977 req-69eb3e2d-23f5-45af-a3f2-fe74edcef187 service nova] Acquiring lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2a5f211b-3c42-47a7-a7c7-b20d14f64977 req-69eb3e2d-23f5-45af-a3f2-fe74edcef187 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2a5f211b-3c42-47a7-a7c7-b20d14f64977 req-69eb3e2d-23f5-45af-a3f2-fe74edcef187 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-2a5f211b-3c42-47a7-a7c7-b20d14f64977 req-69eb3e2d-23f5-45af-a3f2-fe74edcef187 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] No waiting events found dispatching network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:48:58 user nova-compute[72081]: WARNING nova.compute.manager [req-2a5f211b-3c42-47a7-a7c7-b20d14f64977 req-69eb3e2d-23f5-45af-a3f2-fe74edcef187 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received unexpected event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 for instance with vm_state active and task_state deleting.
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-2a5f211b-3c42-47a7-a7c7-b20d14f64977 req-69eb3e2d-23f5-45af-a3f2-fe74edcef187 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received event network-vif-unplugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2a5f211b-3c42-47a7-a7c7-b20d14f64977 req-69eb3e2d-23f5-45af-a3f2-fe74edcef187 service nova] Acquiring lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2a5f211b-3c42-47a7-a7c7-b20d14f64977 req-69eb3e2d-23f5-45af-a3f2-fe74edcef187 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2a5f211b-3c42-47a7-a7c7-b20d14f64977 req-69eb3e2d-23f5-45af-a3f2-fe74edcef187 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-2a5f211b-3c42-47a7-a7c7-b20d14f64977 req-69eb3e2d-23f5-45af-a3f2-fe74edcef187 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] No waiting events found dispatching network-vif-unplugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-2a5f211b-3c42-47a7-a7c7-b20d14f64977 req-69eb3e2d-23f5-45af-a3f2-fe74edcef187 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received event network-vif-unplugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-2a5f211b-3c42-47a7-a7c7-b20d14f64977 req-69eb3e2d-23f5-45af-a3f2-fe74edcef187 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2a5f211b-3c42-47a7-a7c7-b20d14f64977 req-69eb3e2d-23f5-45af-a3f2-fe74edcef187 service nova] Acquiring lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2a5f211b-3c42-47a7-a7c7-b20d14f64977 req-69eb3e2d-23f5-45af-a3f2-fe74edcef187 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2a5f211b-3c42-47a7-a7c7-b20d14f64977 req-69eb3e2d-23f5-45af-a3f2-fe74edcef187 service nova] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-2a5f211b-3c42-47a7-a7c7-b20d14f64977 req-69eb3e2d-23f5-45af-a3f2-fe74edcef187 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] No waiting events found dispatching network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:48:58 user nova-compute[72081]: WARNING nova.compute.manager [req-2a5f211b-3c42-47a7-a7c7-b20d14f64977 req-69eb3e2d-23f5-45af-a3f2-fe74edcef187 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received unexpected event network-vif-plugged-a4385731-05f2-4eac-a5bb-9a987a3f5472 for instance with vm_state active and task_state deleting.
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-2a5f211b-3c42-47a7-a7c7-b20d14f64977 req-69eb3e2d-23f5-45af-a3f2-fe74edcef187 service nova] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Received event network-vif-deleted-a4385731-05f2-4eac-a5bb-9a987a3f5472 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-3323e22e-991e-46e4-ae3e-62498cc2773f req-9b1ae5d4-581e-4dfc-9f94-9cf66fafb757 service nova] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Received event network-vif-plugged-7afd2456-5a12-4283-8760-46d4c7fe8ad1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3323e22e-991e-46e4-ae3e-62498cc2773f req-9b1ae5d4-581e-4dfc-9f94-9cf66fafb757 service nova] Acquiring lock "93a2f40a-1eae-4986-b418-ddbf45723196-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3323e22e-991e-46e4-ae3e-62498cc2773f req-9b1ae5d4-581e-4dfc-9f94-9cf66fafb757 service nova] Lock "93a2f40a-1eae-4986-b418-ddbf45723196-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3323e22e-991e-46e4-ae3e-62498cc2773f req-9b1ae5d4-581e-4dfc-9f94-9cf66fafb757 service nova] Lock "93a2f40a-1eae-4986-b418-ddbf45723196-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-3323e22e-991e-46e4-ae3e-62498cc2773f req-9b1ae5d4-581e-4dfc-9f94-9cf66fafb757 service nova] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] No waiting events found dispatching network-vif-plugged-7afd2456-5a12-4283-8760-46d4c7fe8ad1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:48:58 user nova-compute[72081]: WARNING nova.compute.manager [req-3323e22e-991e-46e4-ae3e-62498cc2773f req-9b1ae5d4-581e-4dfc-9f94-9cf66fafb757 service nova] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Received unexpected event network-vif-plugged-7afd2456-5a12-4283-8760-46d4c7fe8ad1 for instance with vm_state building and task_state spawning.
Apr 12 03:48:58 user nova-compute[72081]: INFO nova.compute.manager [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Took 0.78 seconds to detach 1 volumes for instance.
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271338.229039, 93a2f40a-1eae-4986-b418-ddbf45723196 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:48:58 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] VM Resumed (Lifecycle Event)
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.compute.manager [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:48:58 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Instance spawned successfully.
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:48:58 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271338.2339165, 93a2f40a-1eae-4986-b418-ddbf45723196 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:48:58 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] VM Started (Lifecycle Event)
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:48:58 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:48:58 user nova-compute[72081]: INFO nova.compute.manager [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Took 5.94 seconds to spawn the instance on the hypervisor.
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.compute.manager [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:48:58 user nova-compute[72081]: INFO nova.compute.manager [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Took 6.65 seconds to build instance.
Apr 12 03:48:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dc4ba1cb-4267-4551-9632-9412d9a4255f tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "93a2f40a-1eae-4986-b418-ddbf45723196" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.744s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:48:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.312s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:48:58 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Deleted allocations for instance bcd267d2-5b2c-4ed2-b634-0210fae94dfd
Apr 12 03:48:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8618e686-8881-42a1-8c6c-b96889614ead tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "bcd267d2-5b2c-4ed2-b634-0210fae94dfd" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.401s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:49:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:06 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:11 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681271336.3718386, bcd267d2-5b2c-4ed2-b634-0210fae94dfd => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:49:11 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] VM Stopped (Lifecycle Event)
Apr 12 03:49:11 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b6e4e769-924b-44ed-b367-d2f9dc18c2c2 None None] [instance: bcd267d2-5b2c-4ed2-b634-0210fae94dfd] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:49:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:12 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:49:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:16 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:16 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:49:16 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:49:16 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:49:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:19 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:49:19 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:49:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:49:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:49:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:49:19 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:49:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a6d7d43b-ac65-4ebf-9645-b17e61397076/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a6d7d43b-ac65-4ebf-9645-b17e61397076/disk --force-share --output=json" returned: 0 in 0.189s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a6d7d43b-ac65-4ebf-9645-b17e61397076/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a6d7d43b-ac65-4ebf-9645-b17e61397076/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:49:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:49:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:22 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:49:22 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:49:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=7959MB free_disk=25.889644622802734GB free_vcpus=5 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:49:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:49:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:49:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 2d0904c3-74e0-47bb-a832-758a7035b2f5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:49:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 606425bb-8968-41f9-b3a5-80b6fd188a67 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:49:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 7c2344ea-3ed4-40f5-b318-6fa8588b1fed actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:49:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 12351dbd-0f2e-40a6-a10a-f40f5b6250ef actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:49:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:49:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:49:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance a6d7d43b-ac65-4ebf-9645-b17e61397076 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:49:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 93a2f40a-1eae-4986-b418-ddbf45723196 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:49:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 8 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:49:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=1536MB phys_disk=40GB used_disk=6GB total_vcpus=12 used_vcpus=8 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:49:22 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:49:22 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:49:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:49:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.422s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:49:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:23 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:49:23 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:49:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:49:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-7c2344ea-3ed4-40f5-b318-6fa8588b1fed" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:49:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-7c2344ea-3ed4-40f5-b318-6fa8588b1fed" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:49:23 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 03:49:24 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Updating instance_info_cache with network_info: [{"id": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "address": "fa:16:3e:6d:2b:fe", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.98", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2ac7de8-15", "ovs_interfaceid": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:49:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-7c2344ea-3ed4-40f5-b318-6fa8588b1fed" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:49:24 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 03:49:24 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:49:24 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:49:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:29 user nova-compute[72081]: DEBUG nova.compute.manager [req-3a8502e0-2213-42f2-b0ae-fc36b5196798 req-a9281297-d14d-41ce-9acd-8a5f74ae6469 service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Received event network-changed-2ef93926-6cf3-4d57-be98-ab555ac58b4c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:49:29 user nova-compute[72081]: DEBUG nova.compute.manager [req-3a8502e0-2213-42f2-b0ae-fc36b5196798 req-a9281297-d14d-41ce-9acd-8a5f74ae6469 service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Refreshing instance network info cache due to event network-changed-2ef93926-6cf3-4d57-be98-ab555ac58b4c. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:49:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3a8502e0-2213-42f2-b0ae-fc36b5196798 req-a9281297-d14d-41ce-9acd-8a5f74ae6469 service nova] Acquiring lock "refresh_cache-8d533f5f-dbed-4878-a9b1-2ce4fae8fa13" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:49:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3a8502e0-2213-42f2-b0ae-fc36b5196798 req-a9281297-d14d-41ce-9acd-8a5f74ae6469 service nova] Acquired lock "refresh_cache-8d533f5f-dbed-4878-a9b1-2ce4fae8fa13" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:49:29 user nova-compute[72081]: DEBUG nova.network.neutron [req-3a8502e0-2213-42f2-b0ae-fc36b5196798 req-a9281297-d14d-41ce-9acd-8a5f74ae6469 service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Refreshing network info cache for port 2ef93926-6cf3-4d57-be98-ab555ac58b4c {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:49:29 user nova-compute[72081]: DEBUG nova.network.neutron [req-3a8502e0-2213-42f2-b0ae-fc36b5196798 req-a9281297-d14d-41ce-9acd-8a5f74ae6469 service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Updated VIF entry in instance network info cache for port 2ef93926-6cf3-4d57-be98-ab555ac58b4c. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:49:29 user nova-compute[72081]: DEBUG nova.network.neutron [req-3a8502e0-2213-42f2-b0ae-fc36b5196798 req-a9281297-d14d-41ce-9acd-8a5f74ae6469 service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Updating instance_info_cache with network_info: [{"id": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "address": "fa:16:3e:e2:fd:44", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2ef93926-6c", "ovs_interfaceid": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:49:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3a8502e0-2213-42f2-b0ae-fc36b5196798 req-a9281297-d14d-41ce-9acd-8a5f74ae6469 service nova] Releasing lock "refresh_cache-8d533f5f-dbed-4878-a9b1-2ce4fae8fa13" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:49:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "65ba7373-1427-409d-9aa7-6cb7894d665b" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:49:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:49:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:49:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:49:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:49:33 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:49:33 user nova-compute[72081]: INFO nova.compute.claims [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Claim successful on node user
Apr 12 03:49:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:33 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:49:33 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:49:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.467s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:49:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:49:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:49:33 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:49:33 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:49:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:49:33 user nova-compute[72081]: INFO nova.virt.block_device [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Booting with volume f699cce1-7b7a-47fc-9c5e-161e74d0becd at /dev/vda
Apr 12 03:49:33 user nova-compute[72081]: DEBUG nova.policy [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'a7f422f1a05a4a92ba387e424850b360', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '815b65aa5ad9442db0a087c646f96bbd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:49:33 user nova-compute[72081]: DEBUG os_brick.utils [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:49:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:33 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7d8f8f67-1bda-434e-9933-60705ef4fcc3]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:49:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:33 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[031a81d5-deea-4256-ab89-d39c6773f520]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:49:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:34 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[167e8323-6d6b-4f12-b366-9d999ca36639]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:49:34 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9dab8d0a-e217-4455-a53d-e8611e73d7e1]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:49:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:49:34 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:49:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:49:34 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:49:34 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9b070fec-a6df-4b09-a301-5a0afe38cd19]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:49:34 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:49:34 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:49:34 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:49:34 user nova-compute[72081]: DEBUG os_brick.utils [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] <== get_connector_properties: return (126ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:49:34 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Updating existing volume attachment record: 9b690e24-9152-4eb4-8719-96e77a78ded9 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:49:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:34 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Successfully created port: d7e82a6e-18fe-44f2-b866-f0561d1b8209 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:49:35 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Successfully updated port: d7e82a6e-18fe-44f2-b866-f0561d1b8209 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:49:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "refresh_cache-65ba7373-1427-409d-9aa7-6cb7894d665b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:49:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquired lock "refresh_cache-65ba7373-1427-409d-9aa7-6cb7894d665b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:49:35 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:49:35 user nova-compute[72081]: DEBUG nova.compute.manager [req-539ed144-8436-4419-856a-3d8d5b33c442 req-fe6fe487-a3e9-4b9d-ae28-c233320f53e7 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Received event network-changed-d7e82a6e-18fe-44f2-b866-f0561d1b8209 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:49:35 user nova-compute[72081]: DEBUG nova.compute.manager [req-539ed144-8436-4419-856a-3d8d5b33c442 req-fe6fe487-a3e9-4b9d-ae28-c233320f53e7 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Refreshing instance network info cache due to event network-changed-d7e82a6e-18fe-44f2-b866-f0561d1b8209. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:49:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-539ed144-8436-4419-856a-3d8d5b33c442 req-fe6fe487-a3e9-4b9d-ae28-c233320f53e7 service nova] Acquiring lock "refresh_cache-65ba7373-1427-409d-9aa7-6cb7894d665b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:49:35 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:49:35 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Updating instance_info_cache with network_info: [{"id": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "address": "fa:16:3e:e9:20:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e82a6e-18", "ovs_interfaceid": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:49:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Releasing lock "refresh_cache-65ba7373-1427-409d-9aa7-6cb7894d665b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:49:35 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Instance network_info: |[{"id": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "address": "fa:16:3e:e9:20:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e82a6e-18", "ovs_interfaceid": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:49:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-539ed144-8436-4419-856a-3d8d5b33c442 req-fe6fe487-a3e9-4b9d-ae28-c233320f53e7 service nova] Acquired lock "refresh_cache-65ba7373-1427-409d-9aa7-6cb7894d665b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:49:35 user nova-compute[72081]: DEBUG nova.network.neutron [req-539ed144-8436-4419-856a-3d8d5b33c442 req-fe6fe487-a3e9-4b9d-ae28-c233320f53e7 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Refreshing network info cache for port d7e82a6e-18fe-44f2-b866-f0561d1b8209 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:49:36 user nova-compute[72081]: DEBUG nova.network.neutron [req-539ed144-8436-4419-856a-3d8d5b33c442 req-fe6fe487-a3e9-4b9d-ae28-c233320f53e7 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Updated VIF entry in instance network info cache for port d7e82a6e-18fe-44f2-b866-f0561d1b8209. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:49:36 user nova-compute[72081]: DEBUG nova.network.neutron [req-539ed144-8436-4419-856a-3d8d5b33c442 req-fe6fe487-a3e9-4b9d-ae28-c233320f53e7 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Updating instance_info_cache with network_info: [{"id": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "address": "fa:16:3e:e9:20:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e82a6e-18", "ovs_interfaceid": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:49:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-539ed144-8436-4419-856a-3d8d5b33c442 req-fe6fe487-a3e9-4b9d-ae28-c233320f53e7 service nova] Releasing lock "refresh_cache-65ba7373-1427-409d-9aa7-6cb7894d665b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:49:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.objects.instance [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'flavor' on Instance uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "refresh_cache-606425bb-8968-41f9-b3a5-80b6fd188a67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquired lock "refresh_cache-606425bb-8968-41f9-b3a5-80b6fd188a67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.network.neutron [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.objects.instance [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'info_cache' on Instance uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:49:38 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Creating image(s)
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Did not create local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Ensure instance console log exists: /opt/stack/data/nova/instances/65ba7373-1427-409d-9aa7-6cb7894d665b/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Start _get_guest_xml network_info=[{"id": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "address": "fa:16:3e:e9:20:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e82a6e-18", "ovs_interfaceid": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OVtlqBox', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '65ba7373-1427-409d-9aa7-6cb7894d665b', 'attached_at': '', 'detached_at': '', 'volume_id': 'f699cce1-7b7a-47fc-9c5e-161e74d0becd', 'serial': 'f699cce1-7b7a-47fc-9c5e-161e74d0becd'}, 'disk_bus': 'virtio', 'delete_on_termination': True, 'boot_index': 0, 'guest_format': None, 'device_type': 'disk', 'attachment_id': '9b690e24-9152-4eb4-8719-96e77a78ded9', 'volume_type': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:49:38 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:49:38 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OVtlqBox', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:49:38 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:49:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:49:38 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:49:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:49:38 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ac2400e-85d8-4432-998e-6e28522d19cc]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:49:38 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[58392c7e-4b72-4c6a-ad67-b553393873cb]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd] added
Apr 12 03:49:38 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[19d26c83-258f-4c1f-a856-31d6ac33b149]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d341d8df-47bc-44ea-b68f-291b6cfb6ba3]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op update -n node.session.auth.username -v OVtlqBox {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op update -n node.session.auth.username -v OVtlqBox" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4e96cdb3-d411-4919-b9f2-e00e11ff2bfc]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'OVtlqBox'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[62683d87-e026-4a7c-84a2-ed2cf1b33f63]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e93a6be4-dd34-45ae-973a-416605d6d4a7]: (4, ('tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:49:38 user nova-compute[72081]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 (non-flash)
Apr 12 03:49:38 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsi session list stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:49:38 user nova-compute[72081]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 (non-flash)
Apr 12 03:49:38 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --login" returned: 0 in 0.063s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[92128863-c45d-4d65-a821-3536af9f9cd5]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd, portal: 172.16.0.220,3260]
Apr 12 03:49:38 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd, portal: 172.16.0.220,3260] successful.
Apr 12 03:49:38 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9e1eec60-3f0e-45cf-baf5-09cbf01454f0]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.network.neutron [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Updating instance_info_cache with network_info: [{"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.205", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1ae8bc29-9222-4dac-8523-37962e5721ea]: (4, ('tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:49:38 user nova-compute[72081]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 (non-flash)
Apr 12 03:49:38 user nova-compute[72081]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:49:38 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsi session list stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:49:38 user nova-compute[72081]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 (non-flash)
Apr 12 03:49:38 user nova-compute[72081]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:49:38 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.276s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] HCTL ('33', '-', '-', 0) found on session 41 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Releasing lock "refresh_cache-606425bb-8968-41f9-b3a5-80b6fd188a67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[752a3deb-c5bd-4672-b749-919f8f505cc4]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Searching for a device in session 41 and hctl ['33', '*', '*', 0] yield: None {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:49:38 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Instance destroyed successfully.
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.objects.instance [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'resources' on Instance uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:42:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-2123178219',display_name='tempest-AttachVolumeTestJSON-server-2123178219',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-2123178219',id=31,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDsSrynzjRvdumscunYM7qq2LINQa1dLMdBCKJ6XwNU+wH30BpZ+UxgkL98cPRjcfl2uMCQ+6nM9+XV2FMWC/Uf6P+0R1p0Yeexa689yM1BKJkKUWaai92yYlAVXCdH/1Q==',key_name='tempest-keypair-611227490',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:42:09Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='a341e582192a47548df8627d82c96ea8',ramdisk_id='',reservation_id='r-qag42n01',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-617614045',owner_user_name='tempest-AttachVolumeTestJSON-617614045-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:47:48Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='872fbc3d59274cd480c17b4b7342499a',uuid=606425bb-8968-41f9-b3a5-80b6fd188a67,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.205", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Converting VIF {"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.205", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:85:88:9b,bridge_name='br-int',has_traffic_filtering=True,id=e16af111-f1d4-4f8c-a67a-c80f546c37b1,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16af111-f1') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG os_vif [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:85:88:9b,bridge_name='br-int',has_traffic_filtering=True,id=e16af111-f1d4-4f8c-a67a-c80f546c37b1,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16af111-f1') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tape16af111-f1, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:49:38 user nova-compute[72081]: INFO os_vif [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:85:88:9b,bridge_name='br-int',has_traffic_filtering=True,id=e16af111-f1d4-4f8c-a67a-c80f546c37b1,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16af111-f1')
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Start _get_guest_xml network_info=[{"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.205", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.objects.instance [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'numa_topology' on Instance uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:49:38 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:49:38 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.objects.instance [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'vcpu_model' on Instance uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:42:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-2123178219',display_name='tempest-AttachVolumeTestJSON-server-2123178219',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-2123178219',id=31,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDsSrynzjRvdumscunYM7qq2LINQa1dLMdBCKJ6XwNU+wH30BpZ+UxgkL98cPRjcfl2uMCQ+6nM9+XV2FMWC/Uf6P+0R1p0Yeexa689yM1BKJkKUWaai92yYlAVXCdH/1Q==',key_name='tempest-keypair-611227490',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:42:09Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='a341e582192a47548df8627d82c96ea8',ramdisk_id='',reservation_id='r-qag42n01',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-617614045',owner_user_name='tempest-AttachVolumeTestJSON-617614045-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:47:48Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='872fbc3d59274cd480c17b4b7342499a',uuid=606425bb-8968-41f9-b3a5-80b6fd188a67,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.205", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Converting VIF {"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.205", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:85:88:9b,bridge_name='br-int',has_traffic_filtering=True,id=e16af111-f1d4-4f8c-a67a-c80f546c37b1,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16af111-f1') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.objects.instance [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'pci_devices' on Instance uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:49:38 user nova-compute[72081]:   <uuid>606425bb-8968-41f9-b3a5-80b6fd188a67</uuid>
Apr 12 03:49:38 user nova-compute[72081]:   <name>instance-0000001f</name>
Apr 12 03:49:38 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:49:38 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:49:38 user nova-compute[72081]:   <metadata>
Apr 12 03:49:38 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:49:38 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:49:38 user nova-compute[72081]:       <nova:name>tempest-AttachVolumeTestJSON-server-2123178219</nova:name>
Apr 12 03:49:38 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:49:38</nova:creationTime>
Apr 12 03:49:38 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:49:38 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:49:38 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:49:38 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:49:38 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:49:38 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:49:38 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:49:38 user nova-compute[72081]:       <nova:owner>
Apr 12 03:49:38 user nova-compute[72081]:         <nova:user uuid="872fbc3d59274cd480c17b4b7342499a">tempest-AttachVolumeTestJSON-617614045-project-member</nova:user>
Apr 12 03:49:38 user nova-compute[72081]:         <nova:project uuid="a341e582192a47548df8627d82c96ea8">tempest-AttachVolumeTestJSON-617614045</nova:project>
Apr 12 03:49:38 user nova-compute[72081]:       </nova:owner>
Apr 12 03:49:38 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:49:38 user nova-compute[72081]:       <nova:ports>
Apr 12 03:49:38 user nova-compute[72081]:         <nova:port uuid="e16af111-f1d4-4f8c-a67a-c80f546c37b1">
Apr 12 03:49:38 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Apr 12 03:49:38 user nova-compute[72081]:         </nova:port>
Apr 12 03:49:38 user nova-compute[72081]:       </nova:ports>
Apr 12 03:49:38 user nova-compute[72081]:     </nova:instance>
Apr 12 03:49:38 user nova-compute[72081]:   </metadata>
Apr 12 03:49:38 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:49:38 user nova-compute[72081]:     <system>
Apr 12 03:49:38 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:49:38 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:49:38 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:49:38 user nova-compute[72081]:       <entry name="serial">606425bb-8968-41f9-b3a5-80b6fd188a67</entry>
Apr 12 03:49:38 user nova-compute[72081]:       <entry name="uuid">606425bb-8968-41f9-b3a5-80b6fd188a67</entry>
Apr 12 03:49:38 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:49:38 user nova-compute[72081]:     </system>
Apr 12 03:49:38 user nova-compute[72081]:   </sysinfo>
Apr 12 03:49:38 user nova-compute[72081]:   <os>
Apr 12 03:49:38 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:49:38 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:49:38 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:49:38 user nova-compute[72081]:   </os>
Apr 12 03:49:38 user nova-compute[72081]:   <features>
Apr 12 03:49:38 user nova-compute[72081]:     <acpi/>
Apr 12 03:49:38 user nova-compute[72081]:     <apic/>
Apr 12 03:49:38 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:49:38 user nova-compute[72081]:   </features>
Apr 12 03:49:38 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:49:38 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:49:38 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:49:38 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:49:38 user nova-compute[72081]:   </clock>
Apr 12 03:49:38 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:49:38 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:49:38 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:49:38 user nova-compute[72081]:   </cpu>
Apr 12 03:49:38 user nova-compute[72081]:   <devices>
Apr 12 03:49:38 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:49:38 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:49:38 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk"/>
Apr 12 03:49:38 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:49:38 user nova-compute[72081]:     </disk>
Apr 12 03:49:38 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:49:38 user nova-compute[72081]:       <mac address="fa:16:3e:85:88:9b"/>
Apr 12 03:49:38 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:49:38 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:49:38 user nova-compute[72081]:       <target dev="tape16af111-f1"/>
Apr 12 03:49:38 user nova-compute[72081]:     </interface>
Apr 12 03:49:38 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:49:38 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/console.log" append="off"/>
Apr 12 03:49:38 user nova-compute[72081]:     </serial>
Apr 12 03:49:38 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:49:38 user nova-compute[72081]:     <video>
Apr 12 03:49:38 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:49:38 user nova-compute[72081]:     </video>
Apr 12 03:49:38 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:49:38 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:49:38 user nova-compute[72081]:     </rng>
Apr 12 03:49:38 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:49:38 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:49:38 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:49:38 user nova-compute[72081]:     </memballoon>
Apr 12 03:49:38 user nova-compute[72081]:   </devices>
Apr 12 03:49:38 user nova-compute[72081]: </domain>
Apr 12 03:49:38 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:49:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG nova.objects.instance [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'trusted_certs' on Instance uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Checking if we can resize image /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Cannot resize image /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG nova.objects.instance [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'migration_context' on Instance uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:42:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-2123178219',display_name='tempest-AttachVolumeTestJSON-server-2123178219',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-2123178219',id=31,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDsSrynzjRvdumscunYM7qq2LINQa1dLMdBCKJ6XwNU+wH30BpZ+UxgkL98cPRjcfl2uMCQ+6nM9+XV2FMWC/Uf6P+0R1p0Yeexa689yM1BKJkKUWaai92yYlAVXCdH/1Q==',key_name='tempest-keypair-611227490',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:42:09Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=<?>,power_state=4,progress=0,project_id='a341e582192a47548df8627d82c96ea8',ramdisk_id='',reservation_id='r-qag42n01',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-617614045',owner_user_name='tempest-AttachVolumeTestJSON-617614045-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:47:48Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='872fbc3d59274cd480c17b4b7342499a',uuid=606425bb-8968-41f9-b3a5-80b6fd188a67,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.205", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Converting VIF {"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.205", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:85:88:9b,bridge_name='br-int',has_traffic_filtering=True,id=e16af111-f1d4-4f8c-a67a-c80f546c37b1,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16af111-f1') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG os_vif [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:85:88:9b,bridge_name='br-int',has_traffic_filtering=True,id=e16af111-f1d4-4f8c-a67a-c80f546c37b1,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16af111-f1') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tape16af111-f1, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tape16af111-f1, col_values=(('external_ids', {'iface-id': 'e16af111-f1d4-4f8c-a67a-c80f546c37b1', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:85:88:9b', 'vm-uuid': '606425bb-8968-41f9-b3a5-80b6fd188a67'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:39 user nova-compute[72081]: INFO os_vif [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:85:88:9b,bridge_name='br-int',has_traffic_filtering=True,id=e16af111-f1d4-4f8c-a67a-c80f546c37b1,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16af111-f1')
Apr 12 03:49:39 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Searching for a device in session 41 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OVtlqBox', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.310s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] <== connect_volume: return (1311ms) {'type': 'block', 'scsi_wwn': '23736373732336531', 'path': '/dev/sdb'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23736373732336531', 'path': '/dev/sdb'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:49:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-441149863',display_name='tempest-ServerActionsTestJSON-server-441149863',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-441149863',id=40,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKmrI9wy1CUIwhlKuXvzk0mD7Pt5nZFvI1CsqS0do6Lk76JQOvGl5lvx9rkz7UAM9WGocGXyWSOUOuKTtYZRD2HqKUbTN4VPOoliE1yuu+s0lx7vuZu45Mj7wDb2S8Ksyw==',key_name='tempest-keypair-364276420',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='815b65aa5ad9442db0a087c646f96bbd',ramdisk_id='',reservation_id='r-7tc52gi6',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestJSON-1898059859',owner_user_name='tempest-ServerActionsTestJSON-1898059859-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:49:34Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a7f422f1a05a4a92ba387e424850b360',uuid=65ba7373-1427-409d-9aa7-6cb7894d665b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "address": "fa:16:3e:e9:20:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e82a6e-18", "ovs_interfaceid": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converting VIF {"id": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "address": "fa:16:3e:e9:20:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e82a6e-18", "ovs_interfaceid": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e9:20:ca,bridge_name='br-int',has_traffic_filtering=True,id=d7e82a6e-18fe-44f2-b866-f0561d1b8209,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd7e82a6e-18') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'pci_devices' on Instance uuid 65ba7373-1427-409d-9aa7-6cb7894d665b {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:49:39 user nova-compute[72081]:   <uuid>65ba7373-1427-409d-9aa7-6cb7894d665b</uuid>
Apr 12 03:49:39 user nova-compute[72081]:   <name>instance-00000028</name>
Apr 12 03:49:39 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:49:39 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:49:39 user nova-compute[72081]:   <metadata>
Apr 12 03:49:39 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:49:39 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:49:39 user nova-compute[72081]:       <nova:name>tempest-ServerActionsTestJSON-server-441149863</nova:name>
Apr 12 03:49:39 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:49:38</nova:creationTime>
Apr 12 03:49:39 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:49:39 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:49:39 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:49:39 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:49:39 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:49:39 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:49:39 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:49:39 user nova-compute[72081]:       <nova:owner>
Apr 12 03:49:39 user nova-compute[72081]:         <nova:user uuid="a7f422f1a05a4a92ba387e424850b360">tempest-ServerActionsTestJSON-1898059859-project-member</nova:user>
Apr 12 03:49:39 user nova-compute[72081]:         <nova:project uuid="815b65aa5ad9442db0a087c646f96bbd">tempest-ServerActionsTestJSON-1898059859</nova:project>
Apr 12 03:49:39 user nova-compute[72081]:       </nova:owner>
Apr 12 03:49:39 user nova-compute[72081]:       <nova:ports>
Apr 12 03:49:39 user nova-compute[72081]:         <nova:port uuid="d7e82a6e-18fe-44f2-b866-f0561d1b8209">
Apr 12 03:49:39 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.8" ipVersion="4"/>
Apr 12 03:49:39 user nova-compute[72081]:         </nova:port>
Apr 12 03:49:39 user nova-compute[72081]:       </nova:ports>
Apr 12 03:49:39 user nova-compute[72081]:     </nova:instance>
Apr 12 03:49:39 user nova-compute[72081]:   </metadata>
Apr 12 03:49:39 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:49:39 user nova-compute[72081]:     <system>
Apr 12 03:49:39 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:49:39 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:49:39 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:49:39 user nova-compute[72081]:       <entry name="serial">65ba7373-1427-409d-9aa7-6cb7894d665b</entry>
Apr 12 03:49:39 user nova-compute[72081]:       <entry name="uuid">65ba7373-1427-409d-9aa7-6cb7894d665b</entry>
Apr 12 03:49:39 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:49:39 user nova-compute[72081]:     </system>
Apr 12 03:49:39 user nova-compute[72081]:   </sysinfo>
Apr 12 03:49:39 user nova-compute[72081]:   <os>
Apr 12 03:49:39 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:49:39 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:49:39 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:49:39 user nova-compute[72081]:   </os>
Apr 12 03:49:39 user nova-compute[72081]:   <features>
Apr 12 03:49:39 user nova-compute[72081]:     <acpi/>
Apr 12 03:49:39 user nova-compute[72081]:     <apic/>
Apr 12 03:49:39 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:49:39 user nova-compute[72081]:   </features>
Apr 12 03:49:39 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:49:39 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:49:39 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:49:39 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:49:39 user nova-compute[72081]:   </clock>
Apr 12 03:49:39 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:49:39 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:49:39 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:49:39 user nova-compute[72081]:   </cpu>
Apr 12 03:49:39 user nova-compute[72081]:   <devices>
Apr 12 03:49:39 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:49:39 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:49:39 user nova-compute[72081]:       <source dev="/dev/sdb"/>
Apr 12 03:49:39 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:49:39 user nova-compute[72081]:       <serial>f699cce1-7b7a-47fc-9c5e-161e74d0becd</serial>
Apr 12 03:49:39 user nova-compute[72081]:     </disk>
Apr 12 03:49:39 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:49:39 user nova-compute[72081]:       <mac address="fa:16:3e:e9:20:ca"/>
Apr 12 03:49:39 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:49:39 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:49:39 user nova-compute[72081]:       <target dev="tapd7e82a6e-18"/>
Apr 12 03:49:39 user nova-compute[72081]:     </interface>
Apr 12 03:49:39 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:49:39 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/65ba7373-1427-409d-9aa7-6cb7894d665b/console.log" append="off"/>
Apr 12 03:49:39 user nova-compute[72081]:     </serial>
Apr 12 03:49:39 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:49:39 user nova-compute[72081]:     <video>
Apr 12 03:49:39 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:49:39 user nova-compute[72081]:     </video>
Apr 12 03:49:39 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:49:39 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:49:39 user nova-compute[72081]:     </rng>
Apr 12 03:49:39 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:49:39 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:49:39 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:49:39 user nova-compute[72081]:     </memballoon>
Apr 12 03:49:39 user nova-compute[72081]:   </devices>
Apr 12 03:49:39 user nova-compute[72081]: </domain>
Apr 12 03:49:39 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:49:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-441149863',display_name='tempest-ServerActionsTestJSON-server-441149863',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-441149863',id=40,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKmrI9wy1CUIwhlKuXvzk0mD7Pt5nZFvI1CsqS0do6Lk76JQOvGl5lvx9rkz7UAM9WGocGXyWSOUOuKTtYZRD2HqKUbTN4VPOoliE1yuu+s0lx7vuZu45Mj7wDb2S8Ksyw==',key_name='tempest-keypair-364276420',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='815b65aa5ad9442db0a087c646f96bbd',ramdisk_id='',reservation_id='r-7tc52gi6',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestJSON-1898059859',owner_user_name='tempest-ServerActionsTestJSON-1898059859-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:49:34Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a7f422f1a05a4a92ba387e424850b360',uuid=65ba7373-1427-409d-9aa7-6cb7894d665b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "address": "fa:16:3e:e9:20:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e82a6e-18", "ovs_interfaceid": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converting VIF {"id": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "address": "fa:16:3e:e9:20:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e82a6e-18", "ovs_interfaceid": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e9:20:ca,bridge_name='br-int',has_traffic_filtering=True,id=d7e82a6e-18fe-44f2-b866-f0561d1b8209,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd7e82a6e-18') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG os_vif [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:e9:20:ca,bridge_name='br-int',has_traffic_filtering=True,id=d7e82a6e-18fe-44f2-b866-f0561d1b8209,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd7e82a6e-18') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapd7e82a6e-18, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapd7e82a6e-18, col_values=(('external_ids', {'iface-id': 'd7e82a6e-18fe-44f2-b866-f0561d1b8209', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:e9:20:ca', 'vm-uuid': '65ba7373-1427-409d-9aa7-6cb7894d665b'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:39 user nova-compute[72081]: INFO os_vif [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:e9:20:ca,bridge_name='br-int',has_traffic_filtering=True,id=d7e82a6e-18fe-44f2-b866-f0561d1b8209,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd7e82a6e-18')
Apr 12 03:49:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:49:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] No VIF found with MAC fa:16:3e:e9:20:ca, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:49:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:42 user nova-compute[72081]: DEBUG nova.compute.manager [req-c221637c-f2d7-40df-992d-b9785b2813eb req-8bdc2256-b943-49d1-bc54-8d5fb9c07cbb service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:49:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c221637c-f2d7-40df-992d-b9785b2813eb req-8bdc2256-b943-49d1-bc54-8d5fb9c07cbb service nova] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:49:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c221637c-f2d7-40df-992d-b9785b2813eb req-8bdc2256-b943-49d1-bc54-8d5fb9c07cbb service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:49:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c221637c-f2d7-40df-992d-b9785b2813eb req-8bdc2256-b943-49d1-bc54-8d5fb9c07cbb service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:49:42 user nova-compute[72081]: DEBUG nova.compute.manager [req-c221637c-f2d7-40df-992d-b9785b2813eb req-8bdc2256-b943-49d1-bc54-8d5fb9c07cbb service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] No waiting events found dispatching network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:49:42 user nova-compute[72081]: WARNING nova.compute.manager [req-c221637c-f2d7-40df-992d-b9785b2813eb req-8bdc2256-b943-49d1-bc54-8d5fb9c07cbb service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received unexpected event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 for instance with vm_state stopped and task_state powering-on.
Apr 12 03:49:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.compute.manager [req-7755b02a-c3ea-4c8c-8a03-29a2bc270bbe req-60c78195-a3d5-4bea-a46b-edef597883a3 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7755b02a-c3ea-4c8c-8a03-29a2bc270bbe req-60c78195-a3d5-4bea-a46b-edef597883a3 service nova] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7755b02a-c3ea-4c8c-8a03-29a2bc270bbe req-60c78195-a3d5-4bea-a46b-edef597883a3 service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7755b02a-c3ea-4c8c-8a03-29a2bc270bbe req-60c78195-a3d5-4bea-a46b-edef597883a3 service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.compute.manager [req-7755b02a-c3ea-4c8c-8a03-29a2bc270bbe req-60c78195-a3d5-4bea-a46b-edef597883a3 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] No waiting events found dispatching network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:49:44 user nova-compute[72081]: WARNING nova.compute.manager [req-7755b02a-c3ea-4c8c-8a03-29a2bc270bbe req-60c78195-a3d5-4bea-a46b-edef597883a3 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received unexpected event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 for instance with vm_state stopped and task_state powering-on.
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.compute.manager [req-7755b02a-c3ea-4c8c-8a03-29a2bc270bbe req-60c78195-a3d5-4bea-a46b-edef597883a3 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Received event network-vif-plugged-d7e82a6e-18fe-44f2-b866-f0561d1b8209 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7755b02a-c3ea-4c8c-8a03-29a2bc270bbe req-60c78195-a3d5-4bea-a46b-edef597883a3 service nova] Acquiring lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7755b02a-c3ea-4c8c-8a03-29a2bc270bbe req-60c78195-a3d5-4bea-a46b-edef597883a3 service nova] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7755b02a-c3ea-4c8c-8a03-29a2bc270bbe req-60c78195-a3d5-4bea-a46b-edef597883a3 service nova] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.compute.manager [req-7755b02a-c3ea-4c8c-8a03-29a2bc270bbe req-60c78195-a3d5-4bea-a46b-edef597883a3 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] No waiting events found dispatching network-vif-plugged-d7e82a6e-18fe-44f2-b866-f0561d1b8209 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:49:44 user nova-compute[72081]: WARNING nova.compute.manager [req-7755b02a-c3ea-4c8c-8a03-29a2bc270bbe req-60c78195-a3d5-4bea-a46b-edef597883a3 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Received unexpected event network-vif-plugged-d7e82a6e-18fe-44f2-b866-f0561d1b8209 for instance with vm_state building and task_state spawning.
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.compute.manager [req-7755b02a-c3ea-4c8c-8a03-29a2bc270bbe req-60c78195-a3d5-4bea-a46b-edef597883a3 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Received event network-vif-plugged-d7e82a6e-18fe-44f2-b866-f0561d1b8209 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7755b02a-c3ea-4c8c-8a03-29a2bc270bbe req-60c78195-a3d5-4bea-a46b-edef597883a3 service nova] Acquiring lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7755b02a-c3ea-4c8c-8a03-29a2bc270bbe req-60c78195-a3d5-4bea-a46b-edef597883a3 service nova] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7755b02a-c3ea-4c8c-8a03-29a2bc270bbe req-60c78195-a3d5-4bea-a46b-edef597883a3 service nova] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.compute.manager [req-7755b02a-c3ea-4c8c-8a03-29a2bc270bbe req-60c78195-a3d5-4bea-a46b-edef597883a3 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] No waiting events found dispatching network-vif-plugged-d7e82a6e-18fe-44f2-b866-f0561d1b8209 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:49:44 user nova-compute[72081]: WARNING nova.compute.manager [req-7755b02a-c3ea-4c8c-8a03-29a2bc270bbe req-60c78195-a3d5-4bea-a46b-edef597883a3 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Received unexpected event network-vif-plugged-d7e82a6e-18fe-44f2-b866-f0561d1b8209 for instance with vm_state building and task_state spawning.
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271384.656885, 65ba7373-1427-409d-9aa7-6cb7894d665b => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:49:44 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] VM Resumed (Lifecycle Event)
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:49:44 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Instance spawned successfully.
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:49:44 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Instance rebooted successfully.
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-76538f79-698c-4adf-95c3-9843c2702f12 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:49:44 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271384.6583796, 65ba7373-1427-409d-9aa7-6cb7894d665b => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:49:44 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] VM Started (Lifecycle Event)
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:49:44 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271384.6983004, 606425bb-8968-41f9-b3a5-80b6fd188a67 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:49:44 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] VM Resumed (Lifecycle Event)
Apr 12 03:49:44 user nova-compute[72081]: INFO nova.compute.manager [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Took 6.35 seconds to spawn the instance on the hypervisor.
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271384.7029977, 606425bb-8968-41f9-b3a5-80b6fd188a67 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:49:44 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] VM Started (Lifecycle Event)
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:49:44 user nova-compute[72081]: INFO nova.compute.manager [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Took 11.64 seconds to build instance.
Apr 12 03:49:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4cdaa358-5a81-4c35-9c90-c7d3af72c4ca tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 11.733s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:49:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ff2cbb4-c03e-4073-9b89-0e557dc57842 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "2d0904c3-74e0-47bb-a832-758a7035b2f5" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:49:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ff2cbb4-c03e-4073-9b89-0e557dc57842 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "2d0904c3-74e0-47bb-a832-758a7035b2f5" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:49:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ff2cbb4-c03e-4073-9b89-0e557dc57842 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "2d0904c3-74e0-47bb-a832-758a7035b2f5-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:49:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ff2cbb4-c03e-4073-9b89-0e557dc57842 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "2d0904c3-74e0-47bb-a832-758a7035b2f5-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:49:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ff2cbb4-c03e-4073-9b89-0e557dc57842 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "2d0904c3-74e0-47bb-a832-758a7035b2f5-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:49:45 user nova-compute[72081]: INFO nova.compute.manager [None req-8ff2cbb4-c03e-4073-9b89-0e557dc57842 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Terminating instance
Apr 12 03:49:45 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8ff2cbb4-c03e-4073-9b89-0e557dc57842 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:49:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:46 user nova-compute[72081]: DEBUG nova.compute.manager [req-601feb77-9a23-4a7f-847b-0f38fa5fd86c req-168d6530-6f0a-44f1-a67a-3798a8d05cbf service nova] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Received event network-vif-unplugged-dce84390-a2f8-4047-b513-eec5845a2399 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:49:46 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-601feb77-9a23-4a7f-847b-0f38fa5fd86c req-168d6530-6f0a-44f1-a67a-3798a8d05cbf service nova] Acquiring lock "2d0904c3-74e0-47bb-a832-758a7035b2f5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:49:46 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-601feb77-9a23-4a7f-847b-0f38fa5fd86c req-168d6530-6f0a-44f1-a67a-3798a8d05cbf service nova] Lock "2d0904c3-74e0-47bb-a832-758a7035b2f5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:49:46 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-601feb77-9a23-4a7f-847b-0f38fa5fd86c req-168d6530-6f0a-44f1-a67a-3798a8d05cbf service nova] Lock "2d0904c3-74e0-47bb-a832-758a7035b2f5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:49:46 user nova-compute[72081]: DEBUG nova.compute.manager [req-601feb77-9a23-4a7f-847b-0f38fa5fd86c req-168d6530-6f0a-44f1-a67a-3798a8d05cbf service nova] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] No waiting events found dispatching network-vif-unplugged-dce84390-a2f8-4047-b513-eec5845a2399 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:49:46 user nova-compute[72081]: DEBUG nova.compute.manager [req-601feb77-9a23-4a7f-847b-0f38fa5fd86c req-168d6530-6f0a-44f1-a67a-3798a8d05cbf service nova] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Received event network-vif-unplugged-dce84390-a2f8-4047-b513-eec5845a2399 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:49:46 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Instance destroyed successfully.
Apr 12 03:49:46 user nova-compute[72081]: DEBUG nova.objects.instance [None req-8ff2cbb4-c03e-4073-9b89-0e557dc57842 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lazy-loading 'resources' on Instance uuid 2d0904c3-74e0-47bb-a832-758a7035b2f5 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:49:46 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-8ff2cbb4-c03e-4073-9b89-0e557dc57842 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:30:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1115900831',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1115900831',id=2,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:30:27Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='6e1fa352a8c144a1aada7ce0d8818429',ramdisk_id='',reservation_id='r-zgpfox6y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-590302458',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:32:08Z,user_data=None,user_id='d99be569057f432484e4a60b46882bbd',uuid=2d0904c3-74e0-47bb-a832-758a7035b2f5,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "dce84390-a2f8-4047-b513-eec5845a2399", "address": "fa:16:3e:6e:2e:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdce84390-a2", "ovs_interfaceid": "dce84390-a2f8-4047-b513-eec5845a2399", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:49:46 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8ff2cbb4-c03e-4073-9b89-0e557dc57842 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converting VIF {"id": "dce84390-a2f8-4047-b513-eec5845a2399", "address": "fa:16:3e:6e:2e:db", "network": {"id": "f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1504923357-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6e1fa352a8c144a1aada7ce0d8818429", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdce84390-a2", "ovs_interfaceid": "dce84390-a2f8-4047-b513-eec5845a2399", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:49:46 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8ff2cbb4-c03e-4073-9b89-0e557dc57842 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:6e:2e:db,bridge_name='br-int',has_traffic_filtering=True,id=dce84390-a2f8-4047-b513-eec5845a2399,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdce84390-a2') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:49:46 user nova-compute[72081]: DEBUG os_vif [None req-8ff2cbb4-c03e-4073-9b89-0e557dc57842 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:6e:2e:db,bridge_name='br-int',has_traffic_filtering=True,id=dce84390-a2f8-4047-b513-eec5845a2399,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdce84390-a2') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:49:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapdce84390-a2, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:49:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:49:46 user nova-compute[72081]: INFO os_vif [None req-8ff2cbb4-c03e-4073-9b89-0e557dc57842 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:6e:2e:db,bridge_name='br-int',has_traffic_filtering=True,id=dce84390-a2f8-4047-b513-eec5845a2399,network=Network(f6dd0ec8-a058-4dfb-ac0a-805fe5dc7265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdce84390-a2')
Apr 12 03:49:46 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-8ff2cbb4-c03e-4073-9b89-0e557dc57842 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Deleting instance files /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5_del
Apr 12 03:49:46 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-8ff2cbb4-c03e-4073-9b89-0e557dc57842 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Deletion of /opt/stack/data/nova/instances/2d0904c3-74e0-47bb-a832-758a7035b2f5_del complete
Apr 12 03:49:47 user nova-compute[72081]: INFO nova.compute.manager [None req-8ff2cbb4-c03e-4073-9b89-0e557dc57842 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Took 1.07 seconds to destroy the instance on the hypervisor.
Apr 12 03:49:47 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-8ff2cbb4-c03e-4073-9b89-0e557dc57842 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:49:47 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:49:47 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:49:47 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:49:47 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Took 0.51 seconds to deallocate network for instance.
Apr 12 03:49:47 user nova-compute[72081]: DEBUG nova.compute.manager [req-3655b505-fb81-4cac-9e19-1afe4a2e2e82 req-7f16808f-25d7-4ac1-b8c8-af4978ea4075 service nova] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Received event network-vif-deleted-dce84390-a2f8-4047-b513-eec5845a2399 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:49:47 user nova-compute[72081]: INFO nova.compute.manager [req-3655b505-fb81-4cac-9e19-1afe4a2e2e82 req-7f16808f-25d7-4ac1-b8c8-af4978ea4075 service nova] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Neutron deleted interface dce84390-a2f8-4047-b513-eec5845a2399; detaching it from the instance and deleting it from the info cache
Apr 12 03:49:47 user nova-compute[72081]: DEBUG nova.network.neutron [req-3655b505-fb81-4cac-9e19-1afe4a2e2e82 req-7f16808f-25d7-4ac1-b8c8-af4978ea4075 service nova] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:49:47 user nova-compute[72081]: DEBUG nova.compute.manager [req-3655b505-fb81-4cac-9e19-1afe4a2e2e82 req-7f16808f-25d7-4ac1-b8c8-af4978ea4075 service nova] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Detach interface failed, port_id=dce84390-a2f8-4047-b513-eec5845a2399, reason: Instance 2d0904c3-74e0-47bb-a832-758a7035b2f5 could not be found. {{(pid=72081) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 12 03:49:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ff2cbb4-c03e-4073-9b89-0e557dc57842 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:49:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ff2cbb4-c03e-4073-9b89-0e557dc57842 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:49:47 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-8ff2cbb4-c03e-4073-9b89-0e557dc57842 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:49:47 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-8ff2cbb4-c03e-4073-9b89-0e557dc57842 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:49:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ff2cbb4-c03e-4073-9b89-0e557dc57842 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.325s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:49:47 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-8ff2cbb4-c03e-4073-9b89-0e557dc57842 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Deleted allocations for instance 2d0904c3-74e0-47bb-a832-758a7035b2f5
Apr 12 03:49:48 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8ff2cbb4-c03e-4073-9b89-0e557dc57842 tempest-ServerBootFromVolumeStableRescueTest-590302458 tempest-ServerBootFromVolumeStableRescueTest-590302458-project-member] Lock "2d0904c3-74e0-47bb-a832-758a7035b2f5" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.108s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:49:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:48 user nova-compute[72081]: DEBUG nova.compute.manager [req-d9d1b322-6f73-4ee4-8196-b20d02370780 req-ae1882e3-7995-4f07-a78e-501b48c0eb9e service nova] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Received event network-vif-plugged-dce84390-a2f8-4047-b513-eec5845a2399 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:49:48 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d9d1b322-6f73-4ee4-8196-b20d02370780 req-ae1882e3-7995-4f07-a78e-501b48c0eb9e service nova] Acquiring lock "2d0904c3-74e0-47bb-a832-758a7035b2f5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:49:48 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d9d1b322-6f73-4ee4-8196-b20d02370780 req-ae1882e3-7995-4f07-a78e-501b48c0eb9e service nova] Lock "2d0904c3-74e0-47bb-a832-758a7035b2f5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:49:48 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d9d1b322-6f73-4ee4-8196-b20d02370780 req-ae1882e3-7995-4f07-a78e-501b48c0eb9e service nova] Lock "2d0904c3-74e0-47bb-a832-758a7035b2f5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:49:48 user nova-compute[72081]: DEBUG nova.compute.manager [req-d9d1b322-6f73-4ee4-8196-b20d02370780 req-ae1882e3-7995-4f07-a78e-501b48c0eb9e service nova] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] No waiting events found dispatching network-vif-plugged-dce84390-a2f8-4047-b513-eec5845a2399 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:49:48 user nova-compute[72081]: WARNING nova.compute.manager [req-d9d1b322-6f73-4ee4-8196-b20d02370780 req-ae1882e3-7995-4f07-a78e-501b48c0eb9e service nova] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Received unexpected event network-vif-plugged-dce84390-a2f8-4047-b513-eec5845a2399 for instance with vm_state deleted and task_state None.
Apr 12 03:49:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:49:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:01 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681271386.7496426, 2d0904c3-74e0-47bb-a832-758a7035b2f5 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:50:01 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] VM Stopped (Lifecycle Event)
Apr 12 03:50:01 user nova-compute[72081]: DEBUG nova.compute.manager [None req-98015e71-8f1e-40d3-8a4f-80c7b1ec9960 None None] [instance: 2d0904c3-74e0-47bb-a832-758a7035b2f5] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:50:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:02 user nova-compute[72081]: DEBUG nova.compute.manager [req-4c722a25-7da9-437a-8ad5-f86fc2ca618e req-dd51a32c-dc7b-47a1-adcd-eee59684569b service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Received event network-changed-c06ddd73-9521-41d8-a907-7af844b2adac {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:50:02 user nova-compute[72081]: DEBUG nova.compute.manager [req-4c722a25-7da9-437a-8ad5-f86fc2ca618e req-dd51a32c-dc7b-47a1-adcd-eee59684569b service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Refreshing instance network info cache due to event network-changed-c06ddd73-9521-41d8-a907-7af844b2adac. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:50:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4c722a25-7da9-437a-8ad5-f86fc2ca618e req-dd51a32c-dc7b-47a1-adcd-eee59684569b service nova] Acquiring lock "refresh_cache-6e8d7159-e7c4-4dbd-ab06-074b1d4e3873" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:50:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4c722a25-7da9-437a-8ad5-f86fc2ca618e req-dd51a32c-dc7b-47a1-adcd-eee59684569b service nova] Acquired lock "refresh_cache-6e8d7159-e7c4-4dbd-ab06-074b1d4e3873" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:50:02 user nova-compute[72081]: DEBUG nova.network.neutron [req-4c722a25-7da9-437a-8ad5-f86fc2ca618e req-dd51a32c-dc7b-47a1-adcd-eee59684569b service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Refreshing network info cache for port c06ddd73-9521-41d8-a907-7af844b2adac {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:50:02 user nova-compute[72081]: DEBUG nova.network.neutron [req-4c722a25-7da9-437a-8ad5-f86fc2ca618e req-dd51a32c-dc7b-47a1-adcd-eee59684569b service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Updated VIF entry in instance network info cache for port c06ddd73-9521-41d8-a907-7af844b2adac. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:50:02 user nova-compute[72081]: DEBUG nova.network.neutron [req-4c722a25-7da9-437a-8ad5-f86fc2ca618e req-dd51a32c-dc7b-47a1-adcd-eee59684569b service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Updating instance_info_cache with network_info: [{"id": "c06ddd73-9521-41d8-a907-7af844b2adac", "address": "fa:16:3e:89:fe:e4", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.190", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc06ddd73-95", "ovs_interfaceid": "c06ddd73-9521-41d8-a907-7af844b2adac", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:50:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4c722a25-7da9-437a-8ad5-f86fc2ca618e req-dd51a32c-dc7b-47a1-adcd-eee59684569b service nova] Releasing lock "refresh_cache-6e8d7159-e7c4-4dbd-ab06-074b1d4e3873" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:50:02 user nova-compute[72081]: DEBUG nova.compute.manager [req-35494c42-bdbe-4193-863d-0eb8a22f7342 req-3c0c7673-fe94-4750-9a94-2b44e56677e2 service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Received event network-changed-c06ddd73-9521-41d8-a907-7af844b2adac {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:50:02 user nova-compute[72081]: DEBUG nova.compute.manager [req-35494c42-bdbe-4193-863d-0eb8a22f7342 req-3c0c7673-fe94-4750-9a94-2b44e56677e2 service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Refreshing instance network info cache due to event network-changed-c06ddd73-9521-41d8-a907-7af844b2adac. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:50:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-35494c42-bdbe-4193-863d-0eb8a22f7342 req-3c0c7673-fe94-4750-9a94-2b44e56677e2 service nova] Acquiring lock "refresh_cache-6e8d7159-e7c4-4dbd-ab06-074b1d4e3873" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:50:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-35494c42-bdbe-4193-863d-0eb8a22f7342 req-3c0c7673-fe94-4750-9a94-2b44e56677e2 service nova] Acquired lock "refresh_cache-6e8d7159-e7c4-4dbd-ab06-074b1d4e3873" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:50:02 user nova-compute[72081]: DEBUG nova.network.neutron [req-35494c42-bdbe-4193-863d-0eb8a22f7342 req-3c0c7673-fe94-4750-9a94-2b44e56677e2 service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Refreshing network info cache for port c06ddd73-9521-41d8-a907-7af844b2adac {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "6e8d7159-e7c4-4dbd-ab06-074b1d4e3873" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "6e8d7159-e7c4-4dbd-ab06-074b1d4e3873" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "6e8d7159-e7c4-4dbd-ab06-074b1d4e3873-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "6e8d7159-e7c4-4dbd-ab06-074b1d4e3873-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "6e8d7159-e7c4-4dbd-ab06-074b1d4e3873-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:50:03 user nova-compute[72081]: INFO nova.compute.manager [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Terminating instance
Apr 12 03:50:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG nova.network.neutron [req-35494c42-bdbe-4193-863d-0eb8a22f7342 req-3c0c7673-fe94-4750-9a94-2b44e56677e2 service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Updated VIF entry in instance network info cache for port c06ddd73-9521-41d8-a907-7af844b2adac. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG nova.network.neutron [req-35494c42-bdbe-4193-863d-0eb8a22f7342 req-3c0c7673-fe94-4750-9a94-2b44e56677e2 service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Updating instance_info_cache with network_info: [{"id": "c06ddd73-9521-41d8-a907-7af844b2adac", "address": "fa:16:3e:89:fe:e4", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc06ddd73-95", "ovs_interfaceid": "c06ddd73-9521-41d8-a907-7af844b2adac", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-35494c42-bdbe-4193-863d-0eb8a22f7342 req-3c0c7673-fe94-4750-9a94-2b44e56677e2 service nova] Releasing lock "refresh_cache-6e8d7159-e7c4-4dbd-ab06-074b1d4e3873" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:03 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Instance destroyed successfully.
Apr 12 03:50:03 user nova-compute[72081]: DEBUG nova.objects.instance [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lazy-loading 'resources' on Instance uuid 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:48:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-682928462',display_name='tempest-TestVolumeBootPattern-server-682928462',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-682928462',id=37,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBASnyncVOhHKPzClYx3zwpDcJrieJkBwHOEHWlg2NFTJJdBa1/TdotUOLxC4yYnCsrHxIQy7r8qgZsehiquLQNSYVTnDsRU6DmQUCbMUYEphptEyPo7yf2/NkfI88nssBw==',key_name='tempest-TestVolumeBootPattern-54080024',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:48:20Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='003688353e2f4b36831c574ebaf65173',ramdisk_id='',reservation_id='r-ioaiilyi',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeBootPattern-235043461',owner_user_name='tempest-TestVolumeBootPattern-235043461-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:48:20Z,user_data=None,user_id='b00b348933224558aad1cb12921f100f',uuid=6e8d7159-e7c4-4dbd-ab06-074b1d4e3873,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "c06ddd73-9521-41d8-a907-7af844b2adac", "address": "fa:16:3e:89:fe:e4", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.190", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc06ddd73-95", "ovs_interfaceid": "c06ddd73-9521-41d8-a907-7af844b2adac", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converting VIF {"id": "c06ddd73-9521-41d8-a907-7af844b2adac", "address": "fa:16:3e:89:fe:e4", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.190", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc06ddd73-95", "ovs_interfaceid": "c06ddd73-9521-41d8-a907-7af844b2adac", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:89:fe:e4,bridge_name='br-int',has_traffic_filtering=True,id=c06ddd73-9521-41d8-a907-7af844b2adac,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc06ddd73-95') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG os_vif [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:89:fe:e4,bridge_name='br-int',has_traffic_filtering=True,id=c06ddd73-9521-41d8-a907-7af844b2adac,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc06ddd73-95') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapc06ddd73-95, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:03 user nova-compute[72081]: INFO os_vif [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:89:fe:e4,bridge_name='br-int',has_traffic_filtering=True,id=c06ddd73-9521-41d8-a907-7af844b2adac,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc06ddd73-95')
Apr 12 03:50:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mDcBFWhK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:50:03 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:50:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8dbd4ab1-3384-4f2d-a775-f0829732e427]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:50:03 user nova-compute[72081]: No targets found.
Apr 12 03:50:03 user nova-compute[72081]: iSNS:
Apr 12 03:50:03 user nova-compute[72081]: No targets found.
Apr 12 03:50:03 user nova-compute[72081]: STATIC:
Apr 12 03:50:03 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76
Apr 12 03:50:03 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:50:03 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:50:03 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27
Apr 12 03:50:03 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:50:03 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:50:03 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd
Apr 12 03:50:03 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:50:03 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:50:03 user nova-compute[72081]: FIRMWARE:
Apr 12 03:50:03 user nova-compute[72081]: No targets found.
Apr 12 03:50:03 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:50:03 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:50:03 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0ae72241-81be-4d90-8213-93754a4eecb9]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6fcc599c-f54e-448a-a5bd-95183fd55c51]: (4, ('tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:50:03 user nova-compute[72081]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 (non-flash)
Apr 12 03:50:03 user nova-compute[72081]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:50:03 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsi session list stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:50:03 user nova-compute[72081]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 (non-flash)
Apr 12 03:50:03 user nova-compute[72081]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:50:03 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fb6fab9a0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76'): ({'sdc'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Removing single pathed devices sdc {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[900fab4d-2fa6-488f-b223-9fdd48838636]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e5064b85-0cdb-4b9e-b464-549ea809234b]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Flushing IO for device /dev/sdc {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[44907a77-f7dc-483e-ac1d-b85aeb82a8d1]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:50:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.051s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9615d0dd-e612-4e17-aed8-f8a1a07d1a50]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] SCSI volumes sdc have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-23664303066613765', '/dev/disk/by-id/scsi-SSCST_BIO_Storage_a88b8d80', '/dev/disk/by-id/scsi-1SCST_BIOa88b8d80-6d00fa7e92e75a9e') {{(pid=82083) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3c766b69-fb2a-441e-9cd5-69922a625bf5]: (4, None) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9b92d5d3-d71c-4248-af33-c746d6769ade]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 -p 172.16.0.220:3260 --logout" returned: 0 in 0.053s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[590eda29-e260-4bb5-b8bf-9e4b537e168d]: (4, ('Logging out of session [sid: 40, target: iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76, portal: 172.16.0.220,3260]\nLogout of [sid: 40, target: iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 40, target: iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76, portal: 172.16.0.220,3260]
Apr 12 03:50:04 user nova-compute[72081]: Logout of [sid: 40, target: iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76, portal: 172.16.0.220,3260] successful.
Apr 12 03:50:04 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b27d5f9c-2ca3-4c1d-86b8-9af733bddd76 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[336c82f3-211e-4059-8693-4c6f06176cc6]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.333s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] <== disconnect_volume: return (334ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:50:04 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Deleting instance files /opt/stack/data/nova/instances/6e8d7159-e7c4-4dbd-ab06-074b1d4e3873_del
Apr 12 03:50:04 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Deletion of /opt/stack/data/nova/instances/6e8d7159-e7c4-4dbd-ab06-074b1d4e3873_del complete
Apr 12 03:50:04 user nova-compute[72081]: INFO nova.compute.manager [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Took 1.18 seconds to destroy the instance on the hypervisor.
Apr 12 03:50:04 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:50:04 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Took 0.68 seconds to deallocate network for instance.
Apr 12 03:50:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-86393e8b-e06c-4ff3-9535-96f7a486e251 req-105aa97c-460e-4937-8639-7d8bbb779b65 service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Received event network-vif-deleted-c06ddd73-9521-41d8-a907-7af844b2adac {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-e9a71ab0-809b-4fb5-a85b-c7a46d1942a4 req-13086b21-983d-40e2-bb3b-59e9f3e53ead service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Received event network-vif-unplugged-c06ddd73-9521-41d8-a907-7af844b2adac {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e9a71ab0-809b-4fb5-a85b-c7a46d1942a4 req-13086b21-983d-40e2-bb3b-59e9f3e53ead service nova] Acquiring lock "6e8d7159-e7c4-4dbd-ab06-074b1d4e3873-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e9a71ab0-809b-4fb5-a85b-c7a46d1942a4 req-13086b21-983d-40e2-bb3b-59e9f3e53ead service nova] Lock "6e8d7159-e7c4-4dbd-ab06-074b1d4e3873-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e9a71ab0-809b-4fb5-a85b-c7a46d1942a4 req-13086b21-983d-40e2-bb3b-59e9f3e53ead service nova] Lock "6e8d7159-e7c4-4dbd-ab06-074b1d4e3873-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-e9a71ab0-809b-4fb5-a85b-c7a46d1942a4 req-13086b21-983d-40e2-bb3b-59e9f3e53ead service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] No waiting events found dispatching network-vif-unplugged-c06ddd73-9521-41d8-a907-7af844b2adac {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-e9a71ab0-809b-4fb5-a85b-c7a46d1942a4 req-13086b21-983d-40e2-bb3b-59e9f3e53ead service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Received event network-vif-unplugged-c06ddd73-9521-41d8-a907-7af844b2adac for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-e9a71ab0-809b-4fb5-a85b-c7a46d1942a4 req-13086b21-983d-40e2-bb3b-59e9f3e53ead service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Received event network-vif-plugged-c06ddd73-9521-41d8-a907-7af844b2adac {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e9a71ab0-809b-4fb5-a85b-c7a46d1942a4 req-13086b21-983d-40e2-bb3b-59e9f3e53ead service nova] Acquiring lock "6e8d7159-e7c4-4dbd-ab06-074b1d4e3873-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e9a71ab0-809b-4fb5-a85b-c7a46d1942a4 req-13086b21-983d-40e2-bb3b-59e9f3e53ead service nova] Lock "6e8d7159-e7c4-4dbd-ab06-074b1d4e3873-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e9a71ab0-809b-4fb5-a85b-c7a46d1942a4 req-13086b21-983d-40e2-bb3b-59e9f3e53ead service nova] Lock "6e8d7159-e7c4-4dbd-ab06-074b1d4e3873-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:50:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-e9a71ab0-809b-4fb5-a85b-c7a46d1942a4 req-13086b21-983d-40e2-bb3b-59e9f3e53ead service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] No waiting events found dispatching network-vif-plugged-c06ddd73-9521-41d8-a907-7af844b2adac {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:50:04 user nova-compute[72081]: WARNING nova.compute.manager [req-e9a71ab0-809b-4fb5-a85b-c7a46d1942a4 req-13086b21-983d-40e2-bb3b-59e9f3e53ead service nova] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Received unexpected event network-vif-plugged-c06ddd73-9521-41d8-a907-7af844b2adac for instance with vm_state active and task_state deleting.
Apr 12 03:50:05 user nova-compute[72081]: INFO nova.compute.manager [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Took 0.76 seconds to detach 1 volumes for instance.
Apr 12 03:50:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:50:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:50:05 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:50:05 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:50:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.284s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:50:06 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Deleted allocations for instance 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873
Apr 12 03:50:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f059cc33-d8a6-44e7-a6d4-38d9456bfd6d tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "6e8d7159-e7c4-4dbd-ab06-074b1d4e3873" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.097s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:50:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:14 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:50:16 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:50:17 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:50:17 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:50:18 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681271403.7894845, 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:50:18 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] VM Stopped (Lifecycle Event)
Apr 12 03:50:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-6a894e89-0e2a-420d-b00a-1d02e98763d4 None None] [instance: 6e8d7159-e7c4-4dbd-ab06-074b1d4e3873] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:50:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:50:19 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:50:19 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:50:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:50:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:50:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:50:19 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:50:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:19 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a6d7d43b-ac65-4ebf-9645-b17e61397076/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a6d7d43b-ac65-4ebf-9645-b17e61397076/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a6d7d43b-ac65-4ebf-9645-b17e61397076/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a6d7d43b-ac65-4ebf-9645-b17e61397076/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:50:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:50:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196/disk --force-share --output=json" returned: 0 in 0.154s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.129s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:21 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:50:21 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:50:21 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=7981MB free_disk=25.962299346923828GB free_vcpus=5 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:50:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:50:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:50:21 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 606425bb-8968-41f9-b3a5-80b6fd188a67 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:50:21 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 7c2344ea-3ed4-40f5-b318-6fa8588b1fed actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:50:21 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 12351dbd-0f2e-40a6-a10a-f40f5b6250ef actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:50:21 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:50:21 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance a6d7d43b-ac65-4ebf-9645-b17e61397076 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:50:21 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 93a2f40a-1eae-4986-b418-ddbf45723196 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:50:21 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 65ba7373-1427-409d-9aa7-6cb7894d665b actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:50:21 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 7 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:50:21 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=1408MB phys_disk=40GB used_disk=5GB total_vcpus=12 used_vcpus=7 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:50:22 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:50:22 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:50:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:50:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.389s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:50:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:50:24 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:50:24 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:50:24 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:50:24 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:50:24 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Rebuilding the list of instances to heal {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 12 03:50:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-606425bb-8968-41f9-b3a5-80b6fd188a67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:50:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-606425bb-8968-41f9-b3a5-80b6fd188a67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:50:24 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 03:50:24 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lazy-loading 'info_cache' on Instance uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:50:25 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Updating instance_info_cache with network_info: [{"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.205", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:50:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-606425bb-8968-41f9-b3a5-80b6fd188a67" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:50:25 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 03:50:25 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:50:25 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:50:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-9f5c97fc-58fc-424f-b842-5f5fd6c3b4c1 req-a1a490bf-bf3b-4ee3-9ca9-89d57e7bb45f service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Received event network-changed-7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:50:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-9f5c97fc-58fc-424f-b842-5f5fd6c3b4c1 req-a1a490bf-bf3b-4ee3-9ca9-89d57e7bb45f service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Refreshing instance network info cache due to event network-changed-7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:50:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9f5c97fc-58fc-424f-b842-5f5fd6c3b4c1 req-a1a490bf-bf3b-4ee3-9ca9-89d57e7bb45f service nova] Acquiring lock "refresh_cache-a6d7d43b-ac65-4ebf-9645-b17e61397076" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:50:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9f5c97fc-58fc-424f-b842-5f5fd6c3b4c1 req-a1a490bf-bf3b-4ee3-9ca9-89d57e7bb45f service nova] Acquired lock "refresh_cache-a6d7d43b-ac65-4ebf-9645-b17e61397076" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:50:26 user nova-compute[72081]: DEBUG nova.network.neutron [req-9f5c97fc-58fc-424f-b842-5f5fd6c3b4c1 req-a1a490bf-bf3b-4ee3-9ca9-89d57e7bb45f service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Refreshing network info cache for port 7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:50:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "a6d7d43b-ac65-4ebf-9645-b17e61397076" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:50:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "a6d7d43b-ac65-4ebf-9645-b17e61397076" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:50:26 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lazy-loading 'flavor' on Instance uuid a6d7d43b-ac65-4ebf-9645-b17e61397076 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:50:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "a6d7d43b-ac65-4ebf-9645-b17e61397076" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.056s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:50:26 user nova-compute[72081]: DEBUG nova.network.neutron [req-9f5c97fc-58fc-424f-b842-5f5fd6c3b4c1 req-a1a490bf-bf3b-4ee3-9ca9-89d57e7bb45f service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Updated VIF entry in instance network info cache for port 7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:50:26 user nova-compute[72081]: DEBUG nova.network.neutron [req-9f5c97fc-58fc-424f-b842-5f5fd6c3b4c1 req-a1a490bf-bf3b-4ee3-9ca9-89d57e7bb45f service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Updating instance_info_cache with network_info: [{"id": "7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913", "address": "fa:16:3e:78:3c:6d", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.74", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f861491-1d", "ovs_interfaceid": "7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:50:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9f5c97fc-58fc-424f-b842-5f5fd6c3b4c1 req-a1a490bf-bf3b-4ee3-9ca9-89d57e7bb45f service nova] Releasing lock "refresh_cache-a6d7d43b-ac65-4ebf-9645-b17e61397076" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:50:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "a6d7d43b-ac65-4ebf-9645-b17e61397076" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:50:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "a6d7d43b-ac65-4ebf-9645-b17e61397076" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:50:26 user nova-compute[72081]: INFO nova.compute.manager [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Attaching volume 11ce5801-6746-495f-93bd-4147bd863b6e to /dev/vdb
Apr 12 03:50:27 user nova-compute[72081]: DEBUG os_brick.utils [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:50:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[972e372f-b0c1-43c8-a5a3-ecd5d512fb16]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[180dc0f1-ed66-4736-90fc-70643483bd70]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[070651e1-e1d3-4996-abe4-6f299ead7308]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[85606831-c397-4c6d-ba49-df8e58745f4c]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:50:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:50:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:50:27 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:50:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7b76a070-0fc3-44a2-a028-8fad2855dc8f]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:50:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:50:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:50:27 user nova-compute[72081]: DEBUG os_brick.utils [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] <== get_connector_properties: return (117ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:50:27 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Updating existing volume attachment record: 9f86561d-294d-49fa-b560-69bf9d0fe51f {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:50:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'SFoOxbNk', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:50:31 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:50:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:50:31 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:50:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:50:31 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[97c5c920-e10e-43a6-9880-323fec5b2b15]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:50:31 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[dba0021d-598c-4901-8876-153c1f3d92de]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e] added
Apr 12 03:50:31 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.029s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5793d4bc-ba75-45b0-863b-b1d4cd323e6d]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6aeb0254-05c9-441f-b9d0-b8c787fc06ec]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v SFoOxbNk {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v SFoOxbNk" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[625fc0d2-d150-466f-9238-ebcd52df1a4e]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'SFoOxbNk'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ac76f75a-6089-44b1-ab43-4814f389090e]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[58ca546f-ef9c-4ced-b867-2eb2a3b62deb]: (4, ('tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:50:31 user nova-compute[72081]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:50:31 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsi session list stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:50:31 user nova-compute[72081]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:50:31 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e -p 172.16.0.220:3260 --login" returned: 0 in 0.058s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ce008052-9a2c-4d18-a3dc-38609c8a7701]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e, portal: 172.16.0.220,3260]
Apr 12 03:50:31 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e, portal: 172.16.0.220,3260] successful.
Apr 12 03:50:31 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f88cfff1-2b4b-43fd-a43b-73a05e44e421]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[fa30b62c-ce16-42c2-b7ef-dea14cb08daa]: (4, ('tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)\ntcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:50:31 user nova-compute[72081]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:50:31 user nova-compute[72081]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e (non-flash)
Apr 12 03:50:31 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsi session list stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:50:31 user nova-compute[72081]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:50:31 user nova-compute[72081]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e (non-flash)
Apr 12 03:50:31 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.281s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] HCTL ('34', '-', '-', 0) found on session 42 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d27f51ac-7654-4e73-a867-8bdc8149c704]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Searching for a device in session 42 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:50:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'SFoOxbNk', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:50:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.319s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:50:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] <== connect_volume: return (1321ms) {'type': 'block', 'scsi_wwn': '26131333963626239', 'path': '/dev/sdc'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:50:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26131333963626239', 'path': '/dev/sdc'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:50:32 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lazy-loading 'flavor' on Instance uuid a6d7d43b-ac65-4ebf-9645-b17e61397076 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:50:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] attach device xml: <disk type="block" device="disk">
Apr 12 03:50:32 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:50:32 user nova-compute[72081]:   <source dev="/dev/sdc"/>
Apr 12 03:50:32 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:50:32 user nova-compute[72081]:   <serial>11ce5801-6746-495f-93bd-4147bd863b6e</serial>
Apr 12 03:50:32 user nova-compute[72081]: </disk>
Apr 12 03:50:32 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 03:50:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:50:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:50:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] No VIF found with MAC fa:16:3e:78:3c:6d, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:50:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9d4a616c-bece-4bcf-8ceb-8ee6360e0421 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "a6d7d43b-ac65-4ebf-9645-b17e61397076" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.382s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:50:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "a6d7d43b-ac65-4ebf-9645-b17e61397076" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "a6d7d43b-ac65-4ebf-9645-b17e61397076" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:50:35 user nova-compute[72081]: INFO nova.compute.manager [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Detaching volume 11ce5801-6746-495f-93bd-4147bd863b6e
Apr 12 03:50:35 user nova-compute[72081]: INFO nova.virt.block_device [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Attempting to driver detach volume 11ce5801-6746-495f-93bd-4147bd863b6e from mountpoint /dev/vdb
Apr 12 03:50:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Attempting to detach device vdb from instance a6d7d43b-ac65-4ebf-9645-b17e61397076 from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:50:35 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:50:35 user nova-compute[72081]:   <source dev="/dev/sdc"/>
Apr 12 03:50:35 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:50:35 user nova-compute[72081]:   <serial>11ce5801-6746-495f-93bd-4147bd863b6e</serial>
Apr 12 03:50:35 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 03:50:35 user nova-compute[72081]: </disk>
Apr 12 03:50:35 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:50:35 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Successfully detached device vdb from instance a6d7d43b-ac65-4ebf-9645-b17e61397076 from the persistent domain config.
Apr 12 03:50:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance a6d7d43b-ac65-4ebf-9645-b17e61397076 from the live domain config. {{(pid=72081) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:50:35 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:50:35 user nova-compute[72081]:   <source dev="/dev/sdc"/>
Apr 12 03:50:35 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:50:35 user nova-compute[72081]:   <serial>11ce5801-6746-495f-93bd-4147bd863b6e</serial>
Apr 12 03:50:35 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 03:50:35 user nova-compute[72081]: </disk>
Apr 12 03:50:35 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Received event <DeviceRemovedEvent: 1681271435.577733, a6d7d43b-ac65-4ebf-9645-b17e61397076 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72081) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance a6d7d43b-ac65-4ebf-9645-b17e61397076 {{(pid=72081) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 12 03:50:35 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Successfully detached device vdb from instance a6d7d43b-ac65-4ebf-9645-b17e61397076 from the live domain config.
Apr 12 03:50:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'SFoOxbNk', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:50:35 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:50:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0c104959-66c4-40b8-97a2-084a168d43b2]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:50:35 user nova-compute[72081]: No targets found.
Apr 12 03:50:35 user nova-compute[72081]: iSNS:
Apr 12 03:50:35 user nova-compute[72081]: No targets found.
Apr 12 03:50:35 user nova-compute[72081]: STATIC:
Apr 12 03:50:35 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27
Apr 12 03:50:35 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:50:35 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:50:35 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd
Apr 12 03:50:35 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:50:35 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:50:35 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e
Apr 12 03:50:35 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:50:35 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:50:35 user nova-compute[72081]: FIRMWARE:
Apr 12 03:50:35 user nova-compute[72081]: No targets found.
Apr 12 03:50:35 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:50:35 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:50:35 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2a4bc735-10ee-4ff4-b32a-bda10186a47a]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e07c378a-8988-4e90-a613-f0c8d4fd45e5]: (4, ('tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)\ntcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:50:35 user nova-compute[72081]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:50:35 user nova-compute[72081]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e (non-flash)
Apr 12 03:50:35 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsi session list stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:50:35 user nova-compute[72081]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:50:35 user nova-compute[72081]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e (non-flash)
Apr 12 03:50:35 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3f7443c160>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e'): ({'sdc'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Removing single pathed devices sdc {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[906d0c69-7aca-4ff5-ad8c-3a0a71e6b764]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8ab86fdd-8574-44b8-8d6c-c8aa7d2a96ee]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Flushing IO for device /dev/sdc {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.012s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[576515c7-07a6-4564-a843-d7cd1356c362]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.050s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f8d0268-c6dd-4c4d-b3c5-7f59845c5cdd]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] SCSI volumes sdc have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4f9f0299-1a35-4055-868d-ca56b4b3febd]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e -p 172.16.0.220:3260 --logout" returned: 0 in 0.064s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4c0c3171-648f-47b4-b30f-7fa4c5c3853b]: (4, ('Logging out of session [sid: 42, target: iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e, portal: 172.16.0.220,3260]\nLogout of [sid: 42, target: iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 42, target: iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e, portal: 172.16.0.220,3260]
Apr 12 03:50:35 user nova-compute[72081]: Logout of [sid: 42, target: iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e, portal: 172.16.0.220,3260] successful.
Apr 12 03:50:35 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:11ce5801-6746-495f-93bd-4147bd863b6e -p 172.16.0.220:3260 --op delete" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0dc8eba0-196f-43f9-8129-8a0437eae236]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.303s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] <== disconnect_volume: return (304ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:50:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:50:36 user nova-compute[72081]: DEBUG nova.objects.instance [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lazy-loading 'flavor' on Instance uuid a6d7d43b-ac65-4ebf-9645-b17e61397076 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:50:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-53cc13a2-048d-46a6-bc2f-a33e1e61e14d tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "a6d7d43b-ac65-4ebf-9645-b17e61397076" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.649s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:50:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:42 user nova-compute[72081]: DEBUG nova.compute.manager [req-59bc1da7-1be3-4cf0-8425-05428ff115d2 req-9f20f03a-8adf-4d4d-9b4c-cc04988aae69 service nova] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Received event network-changed-7afd2456-5a12-4283-8760-46d4c7fe8ad1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:50:42 user nova-compute[72081]: DEBUG nova.compute.manager [req-59bc1da7-1be3-4cf0-8425-05428ff115d2 req-9f20f03a-8adf-4d4d-9b4c-cc04988aae69 service nova] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Refreshing instance network info cache due to event network-changed-7afd2456-5a12-4283-8760-46d4c7fe8ad1. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:50:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-59bc1da7-1be3-4cf0-8425-05428ff115d2 req-9f20f03a-8adf-4d4d-9b4c-cc04988aae69 service nova] Acquiring lock "refresh_cache-93a2f40a-1eae-4986-b418-ddbf45723196" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:50:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-59bc1da7-1be3-4cf0-8425-05428ff115d2 req-9f20f03a-8adf-4d4d-9b4c-cc04988aae69 service nova] Acquired lock "refresh_cache-93a2f40a-1eae-4986-b418-ddbf45723196" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:50:42 user nova-compute[72081]: DEBUG nova.network.neutron [req-59bc1da7-1be3-4cf0-8425-05428ff115d2 req-9f20f03a-8adf-4d4d-9b4c-cc04988aae69 service nova] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Refreshing network info cache for port 7afd2456-5a12-4283-8760-46d4c7fe8ad1 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:50:43 user nova-compute[72081]: DEBUG nova.network.neutron [req-59bc1da7-1be3-4cf0-8425-05428ff115d2 req-9f20f03a-8adf-4d4d-9b4c-cc04988aae69 service nova] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Updated VIF entry in instance network info cache for port 7afd2456-5a12-4283-8760-46d4c7fe8ad1. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:50:43 user nova-compute[72081]: DEBUG nova.network.neutron [req-59bc1da7-1be3-4cf0-8425-05428ff115d2 req-9f20f03a-8adf-4d4d-9b4c-cc04988aae69 service nova] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Updating instance_info_cache with network_info: [{"id": "7afd2456-5a12-4283-8760-46d4c7fe8ad1", "address": "fa:16:3e:ba:1d:ce", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.120", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7afd2456-5a", "ovs_interfaceid": "7afd2456-5a12-4283-8760-46d4c7fe8ad1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:50:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-59bc1da7-1be3-4cf0-8425-05428ff115d2 req-9f20f03a-8adf-4d4d-9b4c-cc04988aae69 service nova] Releasing lock "refresh_cache-93a2f40a-1eae-4986-b418-ddbf45723196" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:50:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "93a2f40a-1eae-4986-b418-ddbf45723196" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "93a2f40a-1eae-4986-b418-ddbf45723196" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG nova.objects.instance [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lazy-loading 'flavor' on Instance uuid 93a2f40a-1eae-4986-b418-ddbf45723196 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "93a2f40a-1eae-4986-b418-ddbf45723196" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.063s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "93a2f40a-1eae-4986-b418-ddbf45723196" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "93a2f40a-1eae-4986-b418-ddbf45723196" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:50:44 user nova-compute[72081]: INFO nova.compute.manager [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Attaching volume 35d8d71b-b4ab-44f9-887a-fc760be16530 to /dev/vdb
Apr 12 03:50:44 user nova-compute[72081]: DEBUG os_brick.utils [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f482e802-1905-4d69-a006-5e41ebc4d920]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[947b6a6a-e2b9-4eac-8064-939de9a4818c]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b5d95c6d-cafa-4870-ab0b-6018370fff97]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[fb8c360d-917f-4bb0-a3eb-e105fd05af3f]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:50:44 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:50:44 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ed1361b4-59fb-4d1c-8aba-24cde143fa07]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG os_brick.utils [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] <== get_connector_properties: return (120ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:50:44 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Updating existing volume attachment record: f3ee51ab-e5f5-43ef-9125-9c7edd3f39b8 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:50:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'yAQxkidX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:50:49 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:50:49 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:50:49 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:50:49 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:50:49 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[54e94509-f052-4000-bbaf-ea89237d8bbf]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:50:49 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[52264cab-5bbf-4b51-8dc5-95df2f161a5c]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530] added
Apr 12 03:50:49 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e090f356-237e-446c-ae33-3c6a74f6822c]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[239865ea-aaff-44ea-ae70-36d0fb7dbc5a]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v yAQxkidX {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v yAQxkidX" returned: 0 in 0.034s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1940c39f-0ff4-4f18-88db-112e86ffcf2c]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'yAQxkidX'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[02c4b4cb-702d-410b-97ab-f1ea6ed841aa]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[09f42eae-d444-4678-abca-c8262435023c]: (4, ('tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:50:49 user nova-compute[72081]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:50:49 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsi session list stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:50:49 user nova-compute[72081]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:50:49 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 -p 172.16.0.220:3260 --login" returned: 0 in 0.056s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[df6bf830-3033-4337-b948-d5885354f0cd]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530, portal: 172.16.0.220,3260]
Apr 12 03:50:49 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530, portal: 172.16.0.220,3260] successful.
Apr 12 03:50:49 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d2490ef5-f2b1-49f7-bd8a-7c0b28e4f769]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ed06f32b-cbb4-4387-a0a5-2e1e04b74467]: (4, ('tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:50:49 user nova-compute[72081]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:50:49 user nova-compute[72081]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 (non-flash)
Apr 12 03:50:49 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsi session list stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:50:49 user nova-compute[72081]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:50:49 user nova-compute[72081]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 (non-flash)
Apr 12 03:50:49 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.292s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] HCTL ('34', '-', '-', 0) found on session 43 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ad196d3-d50d-476e-8c08-ce02a03475c1]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Searching for a device in session 43 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:50:49 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'yAQxkidX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:50:50 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.325s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:50:50 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] <== connect_volume: return (1327ms) {'type': 'block', 'scsi_wwn': '26532653865353834', 'path': '/dev/sdc'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:50:50 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26532653865353834', 'path': '/dev/sdc'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:50:50 user nova-compute[72081]: DEBUG nova.objects.instance [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lazy-loading 'flavor' on Instance uuid 93a2f40a-1eae-4986-b418-ddbf45723196 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:50:50 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] attach device xml: <disk type="block" device="disk">
Apr 12 03:50:50 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:50:50 user nova-compute[72081]:   <source dev="/dev/sdc"/>
Apr 12 03:50:50 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:50:50 user nova-compute[72081]:   <serial>35d8d71b-b4ab-44f9-887a-fc760be16530</serial>
Apr 12 03:50:50 user nova-compute[72081]: </disk>
Apr 12 03:50:50 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 03:50:50 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:50:50 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:50:50 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] No VIF found with MAC fa:16:3e:ba:1d:ce, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:50:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-ff6130a1-fa0e-49d1-8a9f-d6a01d2895b7 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "93a2f40a-1eae-4986-b418-ddbf45723196" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.308s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:50:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "93a2f40a-1eae-4986-b418-ddbf45723196" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:50:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "93a2f40a-1eae-4986-b418-ddbf45723196" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:50:51 user nova-compute[72081]: INFO nova.compute.manager [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Detaching volume 35d8d71b-b4ab-44f9-887a-fc760be16530
Apr 12 03:50:51 user nova-compute[72081]: INFO nova.virt.block_device [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Attempting to driver detach volume 35d8d71b-b4ab-44f9-887a-fc760be16530 from mountpoint /dev/vdb
Apr 12 03:50:51 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Attempting to detach device vdb from instance 93a2f40a-1eae-4986-b418-ddbf45723196 from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 03:50:51 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:50:51 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:50:51 user nova-compute[72081]:   <source dev="/dev/sdc"/>
Apr 12 03:50:51 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:50:51 user nova-compute[72081]:   <serial>35d8d71b-b4ab-44f9-887a-fc760be16530</serial>
Apr 12 03:50:51 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 03:50:51 user nova-compute[72081]: </disk>
Apr 12 03:50:51 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:50:51 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Successfully detached device vdb from instance 93a2f40a-1eae-4986-b418-ddbf45723196 from the persistent domain config.
Apr 12 03:50:51 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 93a2f40a-1eae-4986-b418-ddbf45723196 from the live domain config. {{(pid=72081) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 12 03:50:51 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:50:51 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:50:51 user nova-compute[72081]:   <source dev="/dev/sdc"/>
Apr 12 03:50:51 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:50:51 user nova-compute[72081]:   <serial>35d8d71b-b4ab-44f9-887a-fc760be16530</serial>
Apr 12 03:50:51 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 03:50:51 user nova-compute[72081]: </disk>
Apr 12 03:50:51 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Received event <DeviceRemovedEvent: 1681271452.013918, 93a2f40a-1eae-4986-b418-ddbf45723196 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72081) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 93a2f40a-1eae-4986-b418-ddbf45723196 {{(pid=72081) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 12 03:50:52 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Successfully detached device vdb from instance 93a2f40a-1eae-4986-b418-ddbf45723196 from the live domain config.
Apr 12 03:50:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'yAQxkidX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:50:52 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:50:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[96df0784-67ad-42ae-a6ae-c248f632f2b4]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:50:52 user nova-compute[72081]: No targets found.
Apr 12 03:50:52 user nova-compute[72081]: iSNS:
Apr 12 03:50:52 user nova-compute[72081]: No targets found.
Apr 12 03:50:52 user nova-compute[72081]: STATIC:
Apr 12 03:50:52 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27
Apr 12 03:50:52 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:50:52 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:50:52 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd
Apr 12 03:50:52 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:50:52 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:50:52 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530
Apr 12 03:50:52 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:50:52 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:50:52 user nova-compute[72081]: FIRMWARE:
Apr 12 03:50:52 user nova-compute[72081]: No targets found.
Apr 12 03:50:52 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:50:52 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:50:52 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[dfe2f9d8-b98e-4a46-a856-c0e3a3637cc7]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[154587c8-01a1-41c5-9247-74da05dac0ed]: (4, ('tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:50:52 user nova-compute[72081]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:50:52 user nova-compute[72081]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 (non-flash)
Apr 12 03:50:52 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsi session list stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:50:52 user nova-compute[72081]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:50:52 user nova-compute[72081]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 (non-flash)
Apr 12 03:50:52 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3f9416edd0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530'): ({'sdc'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Removing single pathed devices sdc {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f49772a8-9a36-452e-9c09-b4dd68378011]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e342006a-c9e0-4294-9be8-836b450c41e6]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Flushing IO for device /dev/sdc {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9b71f3f7-c47a-4981-a5d8-288ecf36698b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.062s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e1414172-7a16-467d-9220-7ee6dc7638c0]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] SCSI volumes sdc have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[acd68c53-9252-4ffa-8107-12b69e521a14]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 -p 172.16.0.220:3260 --logout" returned: 0 in 0.059s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[30edfdda-0361-4cb0-844f-d3869fe5afc4]: (4, ('Logging out of session [sid: 43, target: iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530, portal: 172.16.0.220,3260]\nLogout of [sid: 43, target: iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 43, target: iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530, portal: 172.16.0.220,3260]
Apr 12 03:50:52 user nova-compute[72081]: Logout of [sid: 43, target: iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530, portal: 172.16.0.220,3260] successful.
Apr 12 03:50:52 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:35d8d71b-b4ab-44f9-887a-fc760be16530 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e1880d2a-2f3d-47c8-a69e-3e8d33dd2526]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.335s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] <== disconnect_volume: return (337ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:50:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:50:53 user nova-compute[72081]: DEBUG nova.objects.instance [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lazy-loading 'flavor' on Instance uuid 93a2f40a-1eae-4986-b418-ddbf45723196 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:50:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f13a81ca-0b61-4144-b171-c25bafdf7436 tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "93a2f40a-1eae-4986-b418-ddbf45723196" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.692s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:50:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-d63a044f-1621-4973-bb22-5382b148da07 req-a0481485-9400-4d5d-9fd2-dfa5aa3eab3c service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Received event network-changed-e1586e63-ee44-4717-ad47-886b1c3f01d0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:50:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-d63a044f-1621-4973-bb22-5382b148da07 req-a0481485-9400-4d5d-9fd2-dfa5aa3eab3c service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Refreshing instance network info cache due to event network-changed-e1586e63-ee44-4717-ad47-886b1c3f01d0. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:50:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d63a044f-1621-4973-bb22-5382b148da07 req-a0481485-9400-4d5d-9fd2-dfa5aa3eab3c service nova] Acquiring lock "refresh_cache-12351dbd-0f2e-40a6-a10a-f40f5b6250ef" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:50:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d63a044f-1621-4973-bb22-5382b148da07 req-a0481485-9400-4d5d-9fd2-dfa5aa3eab3c service nova] Acquired lock "refresh_cache-12351dbd-0f2e-40a6-a10a-f40f5b6250ef" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:50:58 user nova-compute[72081]: DEBUG nova.network.neutron [req-d63a044f-1621-4973-bb22-5382b148da07 req-a0481485-9400-4d5d-9fd2-dfa5aa3eab3c service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Refreshing network info cache for port e1586e63-ee44-4717-ad47-886b1c3f01d0 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:50:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "12351dbd-0f2e-40a6-a10a-f40f5b6250ef" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:50:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "12351dbd-0f2e-40a6-a10a-f40f5b6250ef" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:50:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "12351dbd-0f2e-40a6-a10a-f40f5b6250ef-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:50:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "12351dbd-0f2e-40a6-a10a-f40f5b6250ef-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:50:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "12351dbd-0f2e-40a6-a10a-f40f5b6250ef-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:50:58 user nova-compute[72081]: INFO nova.compute.manager [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Terminating instance
Apr 12 03:50:58 user nova-compute[72081]: DEBUG nova.compute.manager [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:50:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-9bcbf881-7d03-4022-bb80-37530fcb31f9 req-89977fd2-e36b-4d6d-822c-c1c293b90223 service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Received event network-vif-unplugged-e1586e63-ee44-4717-ad47-886b1c3f01d0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:50:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9bcbf881-7d03-4022-bb80-37530fcb31f9 req-89977fd2-e36b-4d6d-822c-c1c293b90223 service nova] Acquiring lock "12351dbd-0f2e-40a6-a10a-f40f5b6250ef-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:50:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9bcbf881-7d03-4022-bb80-37530fcb31f9 req-89977fd2-e36b-4d6d-822c-c1c293b90223 service nova] Lock "12351dbd-0f2e-40a6-a10a-f40f5b6250ef-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:50:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9bcbf881-7d03-4022-bb80-37530fcb31f9 req-89977fd2-e36b-4d6d-822c-c1c293b90223 service nova] Lock "12351dbd-0f2e-40a6-a10a-f40f5b6250ef-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:50:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-9bcbf881-7d03-4022-bb80-37530fcb31f9 req-89977fd2-e36b-4d6d-822c-c1c293b90223 service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] No waiting events found dispatching network-vif-unplugged-e1586e63-ee44-4717-ad47-886b1c3f01d0 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:50:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-9bcbf881-7d03-4022-bb80-37530fcb31f9 req-89977fd2-e36b-4d6d-822c-c1c293b90223 service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Received event network-vif-unplugged-e1586e63-ee44-4717-ad47-886b1c3f01d0 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:50:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG nova.network.neutron [req-d63a044f-1621-4973-bb22-5382b148da07 req-a0481485-9400-4d5d-9fd2-dfa5aa3eab3c service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Updated VIF entry in instance network info cache for port e1586e63-ee44-4717-ad47-886b1c3f01d0. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG nova.network.neutron [req-d63a044f-1621-4973-bb22-5382b148da07 req-a0481485-9400-4d5d-9fd2-dfa5aa3eab3c service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Updating instance_info_cache with network_info: [{"id": "e1586e63-ee44-4717-ad47-886b1c3f01d0", "address": "fa:16:3e:eb:02:66", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1586e63-ee", "ovs_interfaceid": "e1586e63-ee44-4717-ad47-886b1c3f01d0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d63a044f-1621-4973-bb22-5382b148da07 req-a0481485-9400-4d5d-9fd2-dfa5aa3eab3c service nova] Releasing lock "refresh_cache-12351dbd-0f2e-40a6-a10a-f40f5b6250ef" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:59 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Instance destroyed successfully.
Apr 12 03:50:59 user nova-compute[72081]: DEBUG nova.objects.instance [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lazy-loading 'resources' on Instance uuid 12351dbd-0f2e-40a6-a10a-f40f5b6250ef {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:46:12Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1526891958',display_name='tempest-TestVolumeBootPattern-server-1526891958',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1526891958',id=35,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBASnyncVOhHKPzClYx3zwpDcJrieJkBwHOEHWlg2NFTJJdBa1/TdotUOLxC4yYnCsrHxIQy7r8qgZsehiquLQNSYVTnDsRU6DmQUCbMUYEphptEyPo7yf2/NkfI88nssBw==',key_name='tempest-TestVolumeBootPattern-54080024',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:46:23Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='003688353e2f4b36831c574ebaf65173',ramdisk_id='',reservation_id='r-q3m6prjl',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeBootPattern-235043461',owner_user_name='tempest-TestVolumeBootPattern-235043461-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:46:24Z,user_data=None,user_id='b00b348933224558aad1cb12921f100f',uuid=12351dbd-0f2e-40a6-a10a-f40f5b6250ef,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e1586e63-ee44-4717-ad47-886b1c3f01d0", "address": "fa:16:3e:eb:02:66", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.174", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1586e63-ee", "ovs_interfaceid": "e1586e63-ee44-4717-ad47-886b1c3f01d0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converting VIF {"id": "e1586e63-ee44-4717-ad47-886b1c3f01d0", "address": "fa:16:3e:eb:02:66", "network": {"id": "397a8299-6fbd-4bac-97a1-c8cc31fa2e0e", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-175498866-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.174", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "003688353e2f4b36831c574ebaf65173", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1586e63-ee", "ovs_interfaceid": "e1586e63-ee44-4717-ad47-886b1c3f01d0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:eb:02:66,bridge_name='br-int',has_traffic_filtering=True,id=e1586e63-ee44-4717-ad47-886b1c3f01d0,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape1586e63-ee') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG os_vif [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:eb:02:66,bridge_name='br-int',has_traffic_filtering=True,id=e1586e63-ee44-4717-ad47-886b1c3f01d0,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape1586e63-ee') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tape1586e63-ee, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:50:59 user nova-compute[72081]: INFO os_vif [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:eb:02:66,bridge_name='br-int',has_traffic_filtering=True,id=e1586e63-ee44-4717-ad47-886b1c3f01d0,network=Network(397a8299-6fbd-4bac-97a1-c8cc31fa2e0e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape1586e63-ee')
Apr 12 03:50:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wmlvgqFk', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:50:59 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:50:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e03a66e6-fb8c-4279-a034-24a3056bcd4e]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:50:59 user nova-compute[72081]: No targets found.
Apr 12 03:50:59 user nova-compute[72081]: iSNS:
Apr 12 03:50:59 user nova-compute[72081]: No targets found.
Apr 12 03:50:59 user nova-compute[72081]: STATIC:
Apr 12 03:50:59 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27
Apr 12 03:50:59 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:50:59 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:50:59 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd
Apr 12 03:50:59 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:50:59 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:50:59 user nova-compute[72081]: FIRMWARE:
Apr 12 03:50:59 user nova-compute[72081]: No targets found.
Apr 12 03:50:59 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:50:59 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:50:59 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d556eb86-8232-49c3-acdd-d785a69e51c5]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[22810c8b-9c64-4a6e-91e8-3f75780a19d5]: (4, ('tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:50:59 user nova-compute[72081]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:50:59 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsi session list stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 (non-flash)
Apr 12 03:50:59 user nova-compute[72081]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:50:59 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fba3b51b0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27'): ({'sde'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Removing single pathed devices sde {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e5696712-2ed1-4c0a-9aec-31b9de01aa1b]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1f53fa72-af11-48c4-bfec-23cfe6322b2b]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Flushing IO for device /dev/sde {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ad0247ff-2bb5-48cc-b633-98d3e6fb1f32]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.040s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[06546fc7-3902-4dd1-afd9-e35c4a8a5648]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] SCSI volumes sde have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[efacd75f-95f8-4fa7-a728-2d6c17723a1a]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --logout" returned: 0 in 0.057s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[88887fe1-bd20-4db5-ac36-10624b56a2cb]: (4, ('Logging out of session [sid: 37, target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27, portal: 172.16.0.220,3260]\nLogout of [sid: 37, target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 37, target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27, portal: 172.16.0.220,3260]
Apr 12 03:50:59 user nova-compute[72081]: Logout of [sid: 37, target: iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27, portal: 172.16.0.220,3260] successful.
Apr 12 03:50:59 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ac69f59b-6bc4-4f63-bbf6-2952889fcf27 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[079e03ce-b99b-42e7-afe7-452c14ab8df6]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.325s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] <== disconnect_volume: return (327ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:50:59 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Deleting instance files /opt/stack/data/nova/instances/12351dbd-0f2e-40a6-a10a-f40f5b6250ef_del
Apr 12 03:50:59 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Deletion of /opt/stack/data/nova/instances/12351dbd-0f2e-40a6-a10a-f40f5b6250ef_del complete
Apr 12 03:50:59 user nova-compute[72081]: INFO nova.compute.manager [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Took 1.31 seconds to destroy the instance on the hypervisor.
Apr 12 03:50:59 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:50:59 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:51:00 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:51:00 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Took 0.48 seconds to deallocate network for instance.
Apr 12 03:51:00 user nova-compute[72081]: DEBUG nova.compute.manager [req-0b122334-960f-48f8-abaf-c31b731a8b68 req-a960d896-6229-4bbf-879d-3877461a7965 service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Received event network-vif-deleted-e1586e63-ee44-4717-ad47-886b1c3f01d0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:51:00 user nova-compute[72081]: INFO nova.compute.manager [req-0b122334-960f-48f8-abaf-c31b731a8b68 req-a960d896-6229-4bbf-879d-3877461a7965 service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Neutron deleted interface e1586e63-ee44-4717-ad47-886b1c3f01d0; detaching it from the instance and deleting it from the info cache
Apr 12 03:51:00 user nova-compute[72081]: DEBUG nova.network.neutron [req-0b122334-960f-48f8-abaf-c31b731a8b68 req-a960d896-6229-4bbf-879d-3877461a7965 service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:51:00 user nova-compute[72081]: DEBUG nova.compute.manager [req-0b122334-960f-48f8-abaf-c31b731a8b68 req-a960d896-6229-4bbf-879d-3877461a7965 service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Detach interface failed, port_id=e1586e63-ee44-4717-ad47-886b1c3f01d0, reason: Instance 12351dbd-0f2e-40a6-a10a-f40f5b6250ef could not be found. {{(pid=72081) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 12 03:51:00 user nova-compute[72081]: DEBUG nova.compute.manager [req-6bccd904-aa9f-4ddf-a0c6-7dd1b2968f7f req-55640ef2-1633-43e4-961b-8b2115990966 service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Received event network-vif-plugged-e1586e63-ee44-4717-ad47-886b1c3f01d0 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:51:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6bccd904-aa9f-4ddf-a0c6-7dd1b2968f7f req-55640ef2-1633-43e4-961b-8b2115990966 service nova] Acquiring lock "12351dbd-0f2e-40a6-a10a-f40f5b6250ef-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6bccd904-aa9f-4ddf-a0c6-7dd1b2968f7f req-55640ef2-1633-43e4-961b-8b2115990966 service nova] Lock "12351dbd-0f2e-40a6-a10a-f40f5b6250ef-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6bccd904-aa9f-4ddf-a0c6-7dd1b2968f7f req-55640ef2-1633-43e4-961b-8b2115990966 service nova] Lock "12351dbd-0f2e-40a6-a10a-f40f5b6250ef-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:00 user nova-compute[72081]: DEBUG nova.compute.manager [req-6bccd904-aa9f-4ddf-a0c6-7dd1b2968f7f req-55640ef2-1633-43e4-961b-8b2115990966 service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] No waiting events found dispatching network-vif-plugged-e1586e63-ee44-4717-ad47-886b1c3f01d0 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:51:00 user nova-compute[72081]: WARNING nova.compute.manager [req-6bccd904-aa9f-4ddf-a0c6-7dd1b2968f7f req-55640ef2-1633-43e4-961b-8b2115990966 service nova] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Received unexpected event network-vif-plugged-e1586e63-ee44-4717-ad47-886b1c3f01d0 for instance with vm_state active and task_state deleting.
Apr 12 03:51:01 user nova-compute[72081]: INFO nova.compute.manager [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Took 0.77 seconds to detach 1 volumes for instance.
Apr 12 03:51:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:01 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:51:01 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:51:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.239s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:01 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Deleted allocations for instance 12351dbd-0f2e-40a6-a10a-f40f5b6250ef
Apr 12 03:51:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-6723ab86-3cc3-4d6a-bb60-9daa73bb4d48 tempest-TestVolumeBootPattern-235043461 tempest-TestVolumeBootPattern-235043461-project-member] Lock "12351dbd-0f2e-40a6-a10a-f40f5b6250ef" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.986s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:14 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681271459.3254316, 12351dbd-0f2e-40a6-a10a-f40f5b6250ef => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:51:14 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] VM Stopped (Lifecycle Event)
Apr 12 03:51:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b07494ec-d823-47d3-9791-99a0c21df337 None None] [instance: 12351dbd-0f2e-40a6-a10a-f40f5b6250ef] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:51:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:51:16 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:51:16 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:51:17 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:51:17 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Cleaning up deleted instances {{(pid=72081) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 12 03:51:17 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] There are 0 instances to clean {{(pid=72081) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 12 03:51:17 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:51:17 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Cleaning up deleted instances with incomplete migration  {{(pid=72081) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 12 03:51:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:19 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:51:19 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:51:20 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:51:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:20 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:51:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a6d7d43b-ac65-4ebf-9645-b17e61397076/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a6d7d43b-ac65-4ebf-9645-b17e61397076/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a6d7d43b-ac65-4ebf-9645-b17e61397076/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a6d7d43b-ac65-4ebf-9645-b17e61397076/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:51:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67/disk --force-share --output=json" returned: 0 in 0.163s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:22 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:51:22 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:51:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8145MB free_disk=25.950634002685547GB free_vcpus=6 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:51:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:23 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 606425bb-8968-41f9-b3a5-80b6fd188a67 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:51:23 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 7c2344ea-3ed4-40f5-b318-6fa8588b1fed actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:51:23 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:51:23 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance a6d7d43b-ac65-4ebf-9645-b17e61397076 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:51:23 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 93a2f40a-1eae-4986-b418-ddbf45723196 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:51:23 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 65ba7373-1427-409d-9aa7-6cb7894d665b actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:51:23 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 6 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:51:23 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=1280MB phys_disk=40GB used_disk=5GB total_vcpus=12 used_vcpus=6 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:51:23 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Refreshing inventories for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 12 03:51:23 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Updating ProviderTree inventory for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 12 03:51:23 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Updating inventory in ProviderTree for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 12 03:51:23 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Refreshing aggregate associations for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6, aggregates: None {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 12 03:51:23 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Refreshing trait associations for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6, traits: COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_USB,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_RESCUE_BFV,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE41,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_STORAGE_BUS_FDC,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_STORAGE_BUS_IDE,COMPUTE_TRUSTED_CERTS,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_GRAPHICS_MODEL_CIRRUS,HW_CPU_X86_SSSE3,HW_CPU_X86_SSE,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_EXTEND,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_STORAGE_BUS_SATA,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_GRAPHICS_MODEL_QXL,HW_CPU_X86_SSE42,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_ACCELERATORS,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,HW_CPU_X86_MMX,COMPUTE_NODE,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_DEVICE_TAGGING,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_VIOMMU_MODEL_AUTO {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 12 03:51:23 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:51:23 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:51:23 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:51:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.815s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:23 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:51:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:24 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:51:24 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:51:24 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:51:24 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:51:24 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:51:24 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:51:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-7c2344ea-3ed4-40f5-b318-6fa8588b1fed" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:51:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-7c2344ea-3ed4-40f5-b318-6fa8588b1fed" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:51:24 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 03:51:25 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Updating instance_info_cache with network_info: [{"id": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "address": "fa:16:3e:6d:2b:fe", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.98", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2ac7de8-15", "ovs_interfaceid": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:51:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-7c2344ea-3ed4-40f5-b318-6fa8588b1fed" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:51:25 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 03:51:25 user nova-compute[72081]: DEBUG nova.compute.manager [req-bb008382-ee21-42a6-8177-388679f1822d req-1322bf02-0d0c-4f15-8d06-2794afd9b0fe service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Received event network-changed-2ef93926-6cf3-4d57-be98-ab555ac58b4c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:51:25 user nova-compute[72081]: DEBUG nova.compute.manager [req-bb008382-ee21-42a6-8177-388679f1822d req-1322bf02-0d0c-4f15-8d06-2794afd9b0fe service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Refreshing instance network info cache due to event network-changed-2ef93926-6cf3-4d57-be98-ab555ac58b4c. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:51:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bb008382-ee21-42a6-8177-388679f1822d req-1322bf02-0d0c-4f15-8d06-2794afd9b0fe service nova] Acquiring lock "refresh_cache-8d533f5f-dbed-4878-a9b1-2ce4fae8fa13" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:51:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bb008382-ee21-42a6-8177-388679f1822d req-1322bf02-0d0c-4f15-8d06-2794afd9b0fe service nova] Acquired lock "refresh_cache-8d533f5f-dbed-4878-a9b1-2ce4fae8fa13" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:51:25 user nova-compute[72081]: DEBUG nova.network.neutron [req-bb008382-ee21-42a6-8177-388679f1822d req-1322bf02-0d0c-4f15-8d06-2794afd9b0fe service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Refreshing network info cache for port 2ef93926-6cf3-4d57-be98-ab555ac58b4c {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:51:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-4dc72ca9-8eff-4af4-a124-239c1b9f0a7c req-8b82c91a-7e97-4d4e-b167-9c420abdacd3 service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Received event network-changed-7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:51:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-4dc72ca9-8eff-4af4-a124-239c1b9f0a7c req-8b82c91a-7e97-4d4e-b167-9c420abdacd3 service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Refreshing instance network info cache due to event network-changed-7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:51:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4dc72ca9-8eff-4af4-a124-239c1b9f0a7c req-8b82c91a-7e97-4d4e-b167-9c420abdacd3 service nova] Acquiring lock "refresh_cache-a6d7d43b-ac65-4ebf-9645-b17e61397076" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:51:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4dc72ca9-8eff-4af4-a124-239c1b9f0a7c req-8b82c91a-7e97-4d4e-b167-9c420abdacd3 service nova] Acquired lock "refresh_cache-a6d7d43b-ac65-4ebf-9645-b17e61397076" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:51:26 user nova-compute[72081]: DEBUG nova.network.neutron [req-4dc72ca9-8eff-4af4-a124-239c1b9f0a7c req-8b82c91a-7e97-4d4e-b167-9c420abdacd3 service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Refreshing network info cache for port 7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:51:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3dae7b58-3cfb-4af3-ad0d-41ec282b04b8 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "a6d7d43b-ac65-4ebf-9645-b17e61397076" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3dae7b58-3cfb-4af3-ad0d-41ec282b04b8 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "a6d7d43b-ac65-4ebf-9645-b17e61397076" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3dae7b58-3cfb-4af3-ad0d-41ec282b04b8 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "a6d7d43b-ac65-4ebf-9645-b17e61397076-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3dae7b58-3cfb-4af3-ad0d-41ec282b04b8 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "a6d7d43b-ac65-4ebf-9645-b17e61397076-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3dae7b58-3cfb-4af3-ad0d-41ec282b04b8 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "a6d7d43b-ac65-4ebf-9645-b17e61397076-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:26 user nova-compute[72081]: INFO nova.compute.manager [None req-3dae7b58-3cfb-4af3-ad0d-41ec282b04b8 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Terminating instance
Apr 12 03:51:26 user nova-compute[72081]: DEBUG nova.compute.manager [None req-3dae7b58-3cfb-4af3-ad0d-41ec282b04b8 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:51:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-7a38f38d-b3d9-4fdb-9432-883add953c70 req-c6c9e39a-ebe7-4fde-9358-544483d8d2b3 service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Received event network-vif-unplugged-7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:51:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7a38f38d-b3d9-4fdb-9432-883add953c70 req-c6c9e39a-ebe7-4fde-9358-544483d8d2b3 service nova] Acquiring lock "a6d7d43b-ac65-4ebf-9645-b17e61397076-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7a38f38d-b3d9-4fdb-9432-883add953c70 req-c6c9e39a-ebe7-4fde-9358-544483d8d2b3 service nova] Lock "a6d7d43b-ac65-4ebf-9645-b17e61397076-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7a38f38d-b3d9-4fdb-9432-883add953c70 req-c6c9e39a-ebe7-4fde-9358-544483d8d2b3 service nova] Lock "a6d7d43b-ac65-4ebf-9645-b17e61397076-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-7a38f38d-b3d9-4fdb-9432-883add953c70 req-c6c9e39a-ebe7-4fde-9358-544483d8d2b3 service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] No waiting events found dispatching network-vif-unplugged-7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:51:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-7a38f38d-b3d9-4fdb-9432-883add953c70 req-c6c9e39a-ebe7-4fde-9358-544483d8d2b3 service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Received event network-vif-unplugged-7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:51:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:27 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Instance destroyed successfully.
Apr 12 03:51:27 user nova-compute[72081]: DEBUG nova.objects.instance [None req-3dae7b58-3cfb-4af3-ad0d-41ec282b04b8 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lazy-loading 'resources' on Instance uuid a6d7d43b-ac65-4ebf-9645-b17e61397076 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-3dae7b58-3cfb-4af3-ad0d-41ec282b04b8 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:48:33Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestStampPattern-server-2010836582',display_name='tempest-TestStampPattern-server-2010836582',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-teststamppattern-server-2010836582',id=38,image_ref='e36d7fe2-0f9a-43e7-9565-714b8a727ea5',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJxviRdZviloE/Y8vy1de4Nl8WuqpnNhf4wG8IpFYt8u0P1qgKwJzMzVj8wiWWgzOhRDflg4OM3oRDBzEH9LU9bTdJCvVUrt7G9IZqwugvdWPxD2R1lvN0sxiTneX8pngQ==',key_name='tempest-TestStampPattern-474439147',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:48:41Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='c426e52304c14ed9b0c1aed69691bc3d',ramdisk_id='',reservation_id='r-0aywectj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_boot_roles='reader,member',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_image_location='snapshot',image_image_state='available',image_image_type='snapshot',image_instance_uuid='7c2344ea-3ed4-40f5-b318-6fa8588b1fed',image_min_disk='1',image_min_ram='0',image_owner_id='c426e52304c14ed9b0c1aed69691bc3d',image_owner_project_name='tempest-TestStampPattern-707517070',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestStampPattern-707517070-project-member',image_user_id='88c496d8e488434e833e67df25def576',owner_project_name='tempest-TestStampPattern-707517070',owner_user_name='tempest-TestStampPattern-707517070-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:48:41Z,user_data=None,user_id='88c496d8e488434e833e67df25def576',uuid=a6d7d43b-ac65-4ebf-9645-b17e61397076,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913", "address": "fa:16:3e:78:3c:6d", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.74", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f861491-1d", "ovs_interfaceid": "7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-3dae7b58-3cfb-4af3-ad0d-41ec282b04b8 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Converting VIF {"id": "7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913", "address": "fa:16:3e:78:3c:6d", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.74", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f861491-1d", "ovs_interfaceid": "7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-3dae7b58-3cfb-4af3-ad0d-41ec282b04b8 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:78:3c:6d,bridge_name='br-int',has_traffic_filtering=True,id=7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913,network=Network(8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7f861491-1d') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG os_vif [None req-3dae7b58-3cfb-4af3-ad0d-41ec282b04b8 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:78:3c:6d,bridge_name='br-int',has_traffic_filtering=True,id=7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913,network=Network(8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7f861491-1d') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap7f861491-1d, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:51:27 user nova-compute[72081]: INFO os_vif [None req-3dae7b58-3cfb-4af3-ad0d-41ec282b04b8 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:78:3c:6d,bridge_name='br-int',has_traffic_filtering=True,id=7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913,network=Network(8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7f861491-1d')
Apr 12 03:51:27 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-3dae7b58-3cfb-4af3-ad0d-41ec282b04b8 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Deleting instance files /opt/stack/data/nova/instances/a6d7d43b-ac65-4ebf-9645-b17e61397076_del
Apr 12 03:51:27 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-3dae7b58-3cfb-4af3-ad0d-41ec282b04b8 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Deletion of /opt/stack/data/nova/instances/a6d7d43b-ac65-4ebf-9645-b17e61397076_del complete
Apr 12 03:51:27 user nova-compute[72081]: DEBUG nova.network.neutron [req-bb008382-ee21-42a6-8177-388679f1822d req-1322bf02-0d0c-4f15-8d06-2794afd9b0fe service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Updated VIF entry in instance network info cache for port 2ef93926-6cf3-4d57-be98-ab555ac58b4c. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG nova.network.neutron [req-bb008382-ee21-42a6-8177-388679f1822d req-1322bf02-0d0c-4f15-8d06-2794afd9b0fe service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Updating instance_info_cache with network_info: [{"id": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "address": "fa:16:3e:e2:fd:44", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2ef93926-6c", "ovs_interfaceid": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bb008382-ee21-42a6-8177-388679f1822d req-1322bf02-0d0c-4f15-8d06-2794afd9b0fe service nova] Releasing lock "refresh_cache-8d533f5f-dbed-4878-a9b1-2ce4fae8fa13" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:51:27 user nova-compute[72081]: INFO nova.compute.manager [None req-3dae7b58-3cfb-4af3-ad0d-41ec282b04b8 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Took 0.92 seconds to destroy the instance on the hypervisor.
Apr 12 03:51:27 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-3dae7b58-3cfb-4af3-ad0d-41ec282b04b8 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG nova.network.neutron [req-4dc72ca9-8eff-4af4-a124-239c1b9f0a7c req-8b82c91a-7e97-4d4e-b167-9c420abdacd3 service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Updated VIF entry in instance network info cache for port 7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG nova.network.neutron [req-4dc72ca9-8eff-4af4-a124-239c1b9f0a7c req-8b82c91a-7e97-4d4e-b167-9c420abdacd3 service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Updating instance_info_cache with network_info: [{"id": "7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913", "address": "fa:16:3e:78:3c:6d", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f861491-1d", "ovs_interfaceid": "7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4dc72ca9-8eff-4af4-a124-239c1b9f0a7c req-8b82c91a-7e97-4d4e-b167-9c420abdacd3 service nova] Releasing lock "refresh_cache-a6d7d43b-ac65-4ebf-9645-b17e61397076" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Stashing vm_state: active {{(pid=72081) _prep_resize /opt/stack/nova/nova/compute/manager.py:5499}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:51:27 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Took 0.63 seconds to deallocate network for instance.
Apr 12 03:51:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.resize_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.resize_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG nova.objects.instance [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'pci_requests' on Instance uuid 65ba7373-1427-409d-9aa7-6cb7894d665b {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3dae7b58-3cfb-4af3-ad0d-41ec282b04b8 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:51:27 user nova-compute[72081]: INFO nova.compute.claims [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Claim successful on node user
Apr 12 03:51:27 user nova-compute[72081]: DEBUG nova.objects.instance [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'resources' on Instance uuid 65ba7373-1427-409d-9aa7-6cb7894d665b {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG nova.objects.instance [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'pci_devices' on Instance uuid 65ba7373-1427-409d-9aa7-6cb7894d665b {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:51:27 user nova-compute[72081]: INFO nova.compute.resource_tracker [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Updating resource usage from migration 08dbfa70-c3f3-44ff-8dd8-3ea443fb7244
Apr 12 03:51:27 user nova-compute[72081]: DEBUG nova.compute.manager [req-c34a801c-5e2a-461d-ae7b-ad7f39fa5689 req-aca8ee73-1e06-404d-b7a6-343cfcd55af0 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Received event network-changed-d7e82a6e-18fe-44f2-b866-f0561d1b8209 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG nova.compute.manager [req-c34a801c-5e2a-461d-ae7b-ad7f39fa5689 req-aca8ee73-1e06-404d-b7a6-343cfcd55af0 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Refreshing instance network info cache due to event network-changed-d7e82a6e-18fe-44f2-b866-f0561d1b8209. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c34a801c-5e2a-461d-ae7b-ad7f39fa5689 req-aca8ee73-1e06-404d-b7a6-343cfcd55af0 service nova] Acquiring lock "refresh_cache-65ba7373-1427-409d-9aa7-6cb7894d665b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c34a801c-5e2a-461d-ae7b-ad7f39fa5689 req-aca8ee73-1e06-404d-b7a6-343cfcd55af0 service nova] Acquired lock "refresh_cache-65ba7373-1427-409d-9aa7-6cb7894d665b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:51:27 user nova-compute[72081]: DEBUG nova.network.neutron [req-c34a801c-5e2a-461d-ae7b-ad7f39fa5689 req-aca8ee73-1e06-404d-b7a6-343cfcd55af0 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Refreshing network info cache for port d7e82a6e-18fe-44f2-b866-f0561d1b8209 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:51:28 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:51:28 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:51:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.resize_claim" :: held 0.353s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:28 user nova-compute[72081]: INFO nova.compute.manager [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Migrating
Apr 12 03:51:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3dae7b58-3cfb-4af3-ad0d-41ec282b04b8 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.353s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "refresh_cache-65ba7373-1427-409d-9aa7-6cb7894d665b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:51:28 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-3dae7b58-3cfb-4af3-ad0d-41ec282b04b8 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:51:28 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-3dae7b58-3cfb-4af3-ad0d-41ec282b04b8 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:51:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3dae7b58-3cfb-4af3-ad0d-41ec282b04b8 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.324s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:28 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-3dae7b58-3cfb-4af3-ad0d-41ec282b04b8 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Deleted allocations for instance a6d7d43b-ac65-4ebf-9645-b17e61397076
Apr 12 03:51:28 user nova-compute[72081]: DEBUG nova.network.neutron [req-c34a801c-5e2a-461d-ae7b-ad7f39fa5689 req-aca8ee73-1e06-404d-b7a6-343cfcd55af0 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Updated VIF entry in instance network info cache for port d7e82a6e-18fe-44f2-b866-f0561d1b8209. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:51:28 user nova-compute[72081]: DEBUG nova.network.neutron [req-c34a801c-5e2a-461d-ae7b-ad7f39fa5689 req-aca8ee73-1e06-404d-b7a6-343cfcd55af0 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Updating instance_info_cache with network_info: [{"id": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "address": "fa:16:3e:e9:20:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e82a6e-18", "ovs_interfaceid": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:51:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-716d5b53-48c6-4c89-935d-fcc3d924b75a req-a301d001-0176-4489-bab9-1fe38489a9e4 service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Received event network-vif-plugged-7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:51:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-716d5b53-48c6-4c89-935d-fcc3d924b75a req-a301d001-0176-4489-bab9-1fe38489a9e4 service nova] Acquiring lock "a6d7d43b-ac65-4ebf-9645-b17e61397076-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-716d5b53-48c6-4c89-935d-fcc3d924b75a req-a301d001-0176-4489-bab9-1fe38489a9e4 service nova] Lock "a6d7d43b-ac65-4ebf-9645-b17e61397076-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-716d5b53-48c6-4c89-935d-fcc3d924b75a req-a301d001-0176-4489-bab9-1fe38489a9e4 service nova] Lock "a6d7d43b-ac65-4ebf-9645-b17e61397076-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-716d5b53-48c6-4c89-935d-fcc3d924b75a req-a301d001-0176-4489-bab9-1fe38489a9e4 service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] No waiting events found dispatching network-vif-plugged-7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:51:28 user nova-compute[72081]: WARNING nova.compute.manager [req-716d5b53-48c6-4c89-935d-fcc3d924b75a req-a301d001-0176-4489-bab9-1fe38489a9e4 service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Received unexpected event network-vif-plugged-7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913 for instance with vm_state deleted and task_state None.
Apr 12 03:51:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c34a801c-5e2a-461d-ae7b-ad7f39fa5689 req-aca8ee73-1e06-404d-b7a6-343cfcd55af0 service nova] Releasing lock "refresh_cache-65ba7373-1427-409d-9aa7-6cb7894d665b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:51:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-c34a801c-5e2a-461d-ae7b-ad7f39fa5689 req-aca8ee73-1e06-404d-b7a6-343cfcd55af0 service nova] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Received event network-vif-deleted-7f861491-1d4d-4ff0-a1c8-8b0a8ebc7913 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:51:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquired lock "refresh_cache-65ba7373-1427-409d-9aa7-6cb7894d665b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:51:28 user nova-compute[72081]: DEBUG nova.network.neutron [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:51:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-3dae7b58-3cfb-4af3-ad0d-41ec282b04b8 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "a6d7d43b-ac65-4ebf-9645-b17e61397076" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.461s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-af8b5981-3539-4117-aafa-3e5000ab6bb8 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-af8b5981-3539-4117-aafa-3e5000ab6bb8 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-af8b5981-3539-4117-aafa-3e5000ab6bb8 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-af8b5981-3539-4117-aafa-3e5000ab6bb8 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-af8b5981-3539-4117-aafa-3e5000ab6bb8 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:29 user nova-compute[72081]: INFO nova.compute.manager [None req-af8b5981-3539-4117-aafa-3e5000ab6bb8 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Terminating instance
Apr 12 03:51:29 user nova-compute[72081]: DEBUG nova.compute.manager [None req-af8b5981-3539-4117-aafa-3e5000ab6bb8 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:51:29 user nova-compute[72081]: DEBUG nova.network.neutron [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Updating instance_info_cache with network_info: [{"id": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "address": "fa:16:3e:e9:20:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e82a6e-18", "ovs_interfaceid": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:51:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Releasing lock "refresh_cache-65ba7373-1427-409d-9aa7-6cb7894d665b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:51:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Starting migrate_disk_and_power_off {{(pid=72081) migrate_disk_and_power_off /opt/stack/nova/nova/virt/libvirt/driver.py:11460}}
Apr 12 03:51:29 user nova-compute[72081]: DEBUG nova.compute.manager [req-94df2e24-bfec-4f75-9ba9-d7ee54927ad6 req-8d352d35-22ec-4205-96ed-747719dade14 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received event network-vif-unplugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:51:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-94df2e24-bfec-4f75-9ba9-d7ee54927ad6 req-8d352d35-22ec-4205-96ed-747719dade14 service nova] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-94df2e24-bfec-4f75-9ba9-d7ee54927ad6 req-8d352d35-22ec-4205-96ed-747719dade14 service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-94df2e24-bfec-4f75-9ba9-d7ee54927ad6 req-8d352d35-22ec-4205-96ed-747719dade14 service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:29 user nova-compute[72081]: DEBUG nova.compute.manager [req-94df2e24-bfec-4f75-9ba9-d7ee54927ad6 req-8d352d35-22ec-4205-96ed-747719dade14 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] No waiting events found dispatching network-vif-unplugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:51:29 user nova-compute[72081]: DEBUG nova.compute.manager [req-94df2e24-bfec-4f75-9ba9-d7ee54927ad6 req-8d352d35-22ec-4205-96ed-747719dade14 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received event network-vif-unplugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:51:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:30 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Instance destroyed successfully.
Apr 12 03:51:30 user nova-compute[72081]: DEBUG nova.objects.instance [None req-af8b5981-3539-4117-aafa-3e5000ab6bb8 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'resources' on Instance uuid 606425bb-8968-41f9-b3a5-80b6fd188a67 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-af8b5981-3539-4117-aafa-3e5000ab6bb8 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:42:02Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-2123178219',display_name='tempest-AttachVolumeTestJSON-server-2123178219',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-2123178219',id=31,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDsSrynzjRvdumscunYM7qq2LINQa1dLMdBCKJ6XwNU+wH30BpZ+UxgkL98cPRjcfl2uMCQ+6nM9+XV2FMWC/Uf6P+0R1p0Yeexa689yM1BKJkKUWaai92yYlAVXCdH/1Q==',key_name='tempest-keypair-611227490',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:42:09Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='a341e582192a47548df8627d82c96ea8',ramdisk_id='',reservation_id='r-qag42n01',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-617614045',owner_user_name='tempest-AttachVolumeTestJSON-617614045-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:49:45Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='872fbc3d59274cd480c17b4b7342499a',uuid=606425bb-8968-41f9-b3a5-80b6fd188a67,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.205", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-af8b5981-3539-4117-aafa-3e5000ab6bb8 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Converting VIF {"id": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "address": "fa:16:3e:85:88:9b", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.205", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape16af111-f1", "ovs_interfaceid": "e16af111-f1d4-4f8c-a67a-c80f546c37b1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-af8b5981-3539-4117-aafa-3e5000ab6bb8 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:85:88:9b,bridge_name='br-int',has_traffic_filtering=True,id=e16af111-f1d4-4f8c-a67a-c80f546c37b1,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16af111-f1') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG os_vif [None req-af8b5981-3539-4117-aafa-3e5000ab6bb8 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:85:88:9b,bridge_name='br-int',has_traffic_filtering=True,id=e16af111-f1d4-4f8c-a67a-c80f546c37b1,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16af111-f1') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tape16af111-f1, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:30 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Instance destroyed successfully.
Apr 12 03:51:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:49:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-441149863',display_name='tempest-ServerActionsTestJSON-server-441149863',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-441149863',id=40,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKmrI9wy1CUIwhlKuXvzk0mD7Pt5nZFvI1CsqS0do6Lk76JQOvGl5lvx9rkz7UAM9WGocGXyWSOUOuKTtYZRD2HqKUbTN4VPOoliE1yuu+s0lx7vuZu45Mj7wDb2S8Ksyw==',key_name='tempest-keypair-364276420',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:49:44Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=MigrationContext,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='815b65aa5ad9442db0a087c646f96bbd',ramdisk_id='',reservation_id='r-7tc52gi6',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1898059859',owner_user_name='tempest-ServerActionsTestJSON-1898059859-project-member'},tags=<?>,task_state='resize_migrating',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:51:28Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a7f422f1a05a4a92ba387e424850b360',uuid=65ba7373-1427-409d-9aa7-6cb7894d665b,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "address": "fa:16:3e:e9:20:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1452412921-network", "vif_mac": "fa:16:3e:e9:20:ca"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e82a6e-18", "ovs_interfaceid": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converting VIF {"id": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "address": "fa:16:3e:e9:20:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1452412921-network", "vif_mac": "fa:16:3e:e9:20:ca"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e82a6e-18", "ovs_interfaceid": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:e9:20:ca,bridge_name='br-int',has_traffic_filtering=True,id=d7e82a6e-18fe-44f2-b866-f0561d1b8209,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd7e82a6e-18') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG os_vif [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:e9:20:ca,bridge_name='br-int',has_traffic_filtering=True,id=d7e82a6e-18fe-44f2-b866-f0561d1b8209,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd7e82a6e-18') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:51:30 user nova-compute[72081]: INFO os_vif [None req-af8b5981-3539-4117-aafa-3e5000ab6bb8 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:85:88:9b,bridge_name='br-int',has_traffic_filtering=True,id=e16af111-f1d4-4f8c-a67a-c80f546c37b1,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape16af111-f1')
Apr 12 03:51:30 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-af8b5981-3539-4117-aafa-3e5000ab6bb8 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Deleting instance files /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67_del
Apr 12 03:51:30 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-af8b5981-3539-4117-aafa-3e5000ab6bb8 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Deletion of /opt/stack/data/nova/instances/606425bb-8968-41f9-b3a5-80b6fd188a67_del complete
Apr 12 03:51:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapd7e82a6e-18, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:51:30 user nova-compute[72081]: INFO os_vif [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:e9:20:ca,bridge_name='br-int',has_traffic_filtering=True,id=d7e82a6e-18fe-44f2-b866-f0561d1b8209,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd7e82a6e-18')
Apr 12 03:51:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OVtlqBox', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:51:30 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:51:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ac4eedf0-de04-48ea-b311-629ed490f83c]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:51:30 user nova-compute[72081]: No targets found.
Apr 12 03:51:30 user nova-compute[72081]: iSNS:
Apr 12 03:51:30 user nova-compute[72081]: No targets found.
Apr 12 03:51:30 user nova-compute[72081]: STATIC:
Apr 12 03:51:30 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd
Apr 12 03:51:30 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:51:30 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:51:30 user nova-compute[72081]: FIRMWARE:
Apr 12 03:51:30 user nova-compute[72081]: No targets found.
Apr 12 03:51:30 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:51:30 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:51:30 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:30 user nova-compute[72081]: INFO nova.compute.manager [None req-af8b5981-3539-4117-aafa-3e5000ab6bb8 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Took 1.24 seconds to destroy the instance on the hypervisor.
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-af8b5981-3539-4117-aafa-3e5000ab6bb8 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5a1f78e7-5cfc-4902-8ce0-9180a43f097a]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f3cc6064-34aa-438e-b572-699b6676f68f]: (4, ('tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:51:30 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsi session list stdout=tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:51:30 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fb6fa8f70>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd'): ({'sdb'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Removing single pathed devices sdb {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[55f7eb99-0029-4f3d-a92a-ccb4af9a6be9]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7c385d3d-7d9c-4813-b12c-9cb73dca225c]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Flushing IO for device /dev/sdb {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[26a1e82b-de91-4566-b6e9-9fbefa40ded2]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.051s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e77ea4ce-f5d0-44bc-bea3-ce502122506c]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] SCSI volumes sdb have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-23736373732336531', '/dev/disk/by-id/scsi-1SCST_BIOc38f22d1-767723e180fd3f78', '/dev/disk/by-id/scsi-SSCST_BIO_Storage_c38f22d1') {{(pid=82083) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d1003e91-d2a2-4b30-bffb-ae1f4c961ed1]: (4, None) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[58b5584a-817d-4024-8d83-7fd2e8460a24]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --logout" returned: 0 in 0.062s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d89c1c49-dfa0-4159-b21b-4f7b5e1bb423]: (4, ('Logging out of session [sid: 41, target: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd, portal: 172.16.0.220,3260]\nLogout of [sid: 41, target: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 41, target: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd, portal: 172.16.0.220,3260]
Apr 12 03:51:30 user nova-compute[72081]: Logout of [sid: 41, target: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd, portal: 172.16.0.220,3260] successful.
Apr 12 03:51:30 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op delete" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[daacba9f-a247-4a26-b5ef-b9083ba3feeb]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.363s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] <== disconnect_volume: return (365ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG nova.compute.manager [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG nova.compute.manager [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] No waiting events found dispatching network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:51:31 user nova-compute[72081]: WARNING nova.compute.manager [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received unexpected event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 for instance with vm_state active and task_state deleting.
Apr 12 03:51:31 user nova-compute[72081]: DEBUG nova.compute.manager [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Received event network-vif-unplugged-d7e82a6e-18fe-44f2-b866-f0561d1b8209 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] Acquiring lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG nova.compute.manager [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] No waiting events found dispatching network-vif-unplugged-d7e82a6e-18fe-44f2-b866-f0561d1b8209 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:51:31 user nova-compute[72081]: WARNING nova.compute.manager [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Received unexpected event network-vif-unplugged-d7e82a6e-18fe-44f2-b866-f0561d1b8209 for instance with vm_state active and task_state resize_migrating.
Apr 12 03:51:31 user nova-compute[72081]: DEBUG nova.compute.manager [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Received event network-vif-plugged-d7e82a6e-18fe-44f2-b866-f0561d1b8209 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] Acquiring lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG nova.compute.manager [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] No waiting events found dispatching network-vif-plugged-d7e82a6e-18fe-44f2-b866-f0561d1b8209 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:51:31 user nova-compute[72081]: WARNING nova.compute.manager [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Received unexpected event network-vif-plugged-d7e82a6e-18fe-44f2-b866-f0561d1b8209 for instance with vm_state active and task_state resize_migrating.
Apr 12 03:51:31 user nova-compute[72081]: DEBUG nova.compute.manager [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG nova.compute.manager [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] No waiting events found dispatching network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:51:31 user nova-compute[72081]: WARNING nova.compute.manager [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received unexpected event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 for instance with vm_state active and task_state deleting.
Apr 12 03:51:31 user nova-compute[72081]: DEBUG nova.compute.manager [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG nova.compute.manager [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] No waiting events found dispatching network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:51:31 user nova-compute[72081]: WARNING nova.compute.manager [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received unexpected event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 for instance with vm_state active and task_state deleting.
Apr 12 03:51:31 user nova-compute[72081]: DEBUG nova.compute.manager [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received event network-vif-unplugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG nova.compute.manager [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] No waiting events found dispatching network-vif-unplugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG nova.compute.manager [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received event network-vif-unplugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG nova.compute.manager [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] Acquiring lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG nova.compute.manager [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] No waiting events found dispatching network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:51:31 user nova-compute[72081]: WARNING nova.compute.manager [req-7076d93b-fdaf-4477-bc74-372c4ef7b638 req-aa28b49b-3f0f-4665-8420-2f1e7f5acfd7 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received unexpected event network-vif-plugged-e16af111-f1d4-4f8c-a67a-c80f546c37b1 for instance with vm_state active and task_state deleting.
Apr 12 03:51:31 user nova-compute[72081]: DEBUG nova.network.neutron [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Port d7e82a6e-18fe-44f2-b866-f0561d1b8209 binding to destination host user is already ACTIVE {{(pid=72081) migrate_instance_start /opt/stack/nova/nova/network/neutron.py:3161}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:51:31 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Took 1.49 seconds to deallocate network for instance.
Apr 12 03:51:31 user nova-compute[72081]: DEBUG nova.compute.manager [req-c7c08b5c-3a51-4f20-81f5-12c2d3159e27 req-ebec080a-1cae-4492-8686-6fc331aa0658 service nova] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Received event network-vif-deleted-e16af111-f1d4-4f8c-a67a-c80f546c37b1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-af8b5981-3539-4117-aafa-3e5000ab6bb8 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-af8b5981-3539-4117-aafa-3e5000ab6bb8 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "refresh_cache-65ba7373-1427-409d-9aa7-6cb7894d665b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquired lock "refresh_cache-65ba7373-1427-409d-9aa7-6cb7894d665b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG nova.network.neutron [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-af8b5981-3539-4117-aafa-3e5000ab6bb8 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-af8b5981-3539-4117-aafa-3e5000ab6bb8 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-af8b5981-3539-4117-aafa-3e5000ab6bb8 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.226s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:32 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-af8b5981-3539-4117-aafa-3e5000ab6bb8 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Deleted allocations for instance 606425bb-8968-41f9-b3a5-80b6fd188a67
Apr 12 03:51:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-af8b5981-3539-4117-aafa-3e5000ab6bb8 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "606425bb-8968-41f9-b3a5-80b6fd188a67" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.166s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG nova.network.neutron [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Updating instance_info_cache with network_info: [{"id": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "address": "fa:16:3e:e9:20:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e82a6e-18", "ovs_interfaceid": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Releasing lock "refresh_cache-65ba7373-1427-409d-9aa7-6cb7894d665b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG os_brick.utils [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9e481ac2-6436-475c-9f0d-b60d4d99f524]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ad444401-792c-4db5-8d4a-149507a3032f]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[dfe80e99-8354-4d0c-9fc6-920b408a7432]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c8b715fa-ad5b-475e-af8b-d1bf65b92307]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:51:32 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:51:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4ce4c171-b2f8-4a6c-8eef-acdde5dee3d7]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:51:32 user nova-compute[72081]: DEBUG os_brick.utils [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] <== get_connector_properties: return (116ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:51:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Starting finish_migration {{(pid=72081) finish_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11647}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Instance directory exists: not creating {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4694}}
Apr 12 03:51:37 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Creating image(s)
Apr 12 03:51:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Did not create local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Ensure instance console log exists: /opt/stack/data/nova/instances/65ba7373-1427-409d-9aa7-6cb7894d665b/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Start _get_guest_xml network_info=[{"id": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "address": "fa:16:3e:e9:20:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1452412921-network", "vif_mac": "fa:16:3e:e9:20:ca"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e82a6e-18", "ovs_interfaceid": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OVtlqBox', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'attaching', 'instance': '65ba7373-1427-409d-9aa7-6cb7894d665b', 'attached_at': '2023-04-12T03:51:37.000000', 'detached_at': '', 'volume_id': 'f699cce1-7b7a-47fc-9c5e-161e74d0becd', 'serial': 'f699cce1-7b7a-47fc-9c5e-161e74d0becd'}, 'disk_bus': 'virtio', 'delete_on_termination': True, 'boot_index': 0, 'guest_format': None, 'device_type': 'disk', 'attachment_id': 'b6442731-a335-4b7d-b2fc-12da423f840a', 'volume_type': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:51:37 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:51:37 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:51:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:31Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='84',id=12,is_public=True,memory_mb=192,name='m1.micro',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG nova.objects.instance [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'vcpu_model' on Instance uuid 65ba7373-1427-409d-9aa7-6cb7894d665b {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OVtlqBox', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:51:37 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:51:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:51:37 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:51:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:51:37 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7b164a04-7cc6-49c7-9f71-57814eed16c0]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:51:37 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d352d3a7-5b85-43f3-955c-1e0684f7b882]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd] added
Apr 12 03:51:37 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b1779d58-8184-4a44-8b2e-9d07fe00b2de]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.033s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[896623c4-4d3e-49a7-825c-f01043393c48]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op update -n node.session.auth.username -v OVtlqBox {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op update -n node.session.auth.username -v OVtlqBox" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[00393936-f98b-45de-9255-6e0df902f32b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'OVtlqBox'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6f1df15-bbb1-4a6d-9c8c-15985bd56614]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[688e39a0-840e-483a-94ae-9f7836128c96]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Apr 12 03:51:37 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Apr 12 03:51:37 user nova-compute[72081]:  {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:51:37 user nova-compute[72081]: WARNING os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Apr 12 03:51:37 user nova-compute[72081]: 
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --login" returned: 0 in 0.045s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f150c53d-6f85-4d38-85ef-30bf6d4c80fa]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd, portal: 172.16.0.220,3260]
Apr 12 03:51:37 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd, portal: 172.16.0.220,3260] successful.
Apr 12 03:51:37 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[783edb1d-5300-46aa-bd82-490ddb535197]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bcd68e41-76c0-4936-8084-f9a9b0aaf9a5]: (4, ('tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:51:37 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsi session list stdout=tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:51:37 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.266s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] HCTL ('33', '-', '-', 0) found on session 44 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb837f0a-8a04-4d0c-a8bc-23aeec567cc8]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Searching for a device in session 44 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:51:37 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OVtlqBox', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:51:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.302s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:51:38 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] <== connect_volume: return (1303ms) {'type': 'block', 'scsi_wwn': '23736373732336531', 'path': '/dev/sdb'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:51:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23736373732336531', 'path': '/dev/sdb'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:51:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:49:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-441149863',display_name='tempest-ServerActionsTestJSON-server-441149863',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(12),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-441149863',id=40,image_ref='',info_cache=InstanceInfoCache,instance_type_id=12,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKmrI9wy1CUIwhlKuXvzk0mD7Pt5nZFvI1CsqS0do6Lk76JQOvGl5lvx9rkz7UAM9WGocGXyWSOUOuKTtYZRD2HqKUbTN4VPOoliE1yuu+s0lx7vuZu45Mj7wDb2S8Ksyw==',key_name='tempest-keypair-364276420',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:49:44Z,launched_on='user',locked=False,locked_by=None,memory_mb=192,metadata={},migration_context=MigrationContext,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=Flavor(11),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='815b65aa5ad9442db0a087c646f96bbd',ramdisk_id='',reservation_id='r-7tc52gi6',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1898059859',owner_user_name='tempest-ServerActionsTestJSON-1898059859-project-member'},tags=<?>,task_state='resize_finish',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:51:32Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a7f422f1a05a4a92ba387e424850b360',uuid=65ba7373-1427-409d-9aa7-6cb7894d665b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "address": "fa:16:3e:e9:20:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1452412921-network", "vif_mac": "fa:16:3e:e9:20:ca"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e82a6e-18", "ovs_interfaceid": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:51:38 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converting VIF {"id": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "address": "fa:16:3e:e9:20:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1452412921-network", "vif_mac": "fa:16:3e:e9:20:ca"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e82a6e-18", "ovs_interfaceid": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:51:38 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e9:20:ca,bridge_name='br-int',has_traffic_filtering=True,id=d7e82a6e-18fe-44f2-b866-f0561d1b8209,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd7e82a6e-18') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:51:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:51:38 user nova-compute[72081]:   <uuid>65ba7373-1427-409d-9aa7-6cb7894d665b</uuid>
Apr 12 03:51:38 user nova-compute[72081]:   <name>instance-00000028</name>
Apr 12 03:51:38 user nova-compute[72081]:   <memory>196608</memory>
Apr 12 03:51:38 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:51:38 user nova-compute[72081]:   <metadata>
Apr 12 03:51:38 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:51:38 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:51:38 user nova-compute[72081]:       <nova:name>tempest-ServerActionsTestJSON-server-441149863</nova:name>
Apr 12 03:51:38 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:51:37</nova:creationTime>
Apr 12 03:51:38 user nova-compute[72081]:       <nova:flavor name="m1.micro">
Apr 12 03:51:38 user nova-compute[72081]:         <nova:memory>192</nova:memory>
Apr 12 03:51:38 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:51:38 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:51:38 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:51:38 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:51:38 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:51:38 user nova-compute[72081]:       <nova:owner>
Apr 12 03:51:38 user nova-compute[72081]:         <nova:user uuid="a7f422f1a05a4a92ba387e424850b360">tempest-ServerActionsTestJSON-1898059859-project-member</nova:user>
Apr 12 03:51:38 user nova-compute[72081]:         <nova:project uuid="815b65aa5ad9442db0a087c646f96bbd">tempest-ServerActionsTestJSON-1898059859</nova:project>
Apr 12 03:51:38 user nova-compute[72081]:       </nova:owner>
Apr 12 03:51:38 user nova-compute[72081]:       <nova:ports>
Apr 12 03:51:38 user nova-compute[72081]:         <nova:port uuid="d7e82a6e-18fe-44f2-b866-f0561d1b8209">
Apr 12 03:51:38 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.8" ipVersion="4"/>
Apr 12 03:51:38 user nova-compute[72081]:         </nova:port>
Apr 12 03:51:38 user nova-compute[72081]:       </nova:ports>
Apr 12 03:51:38 user nova-compute[72081]:     </nova:instance>
Apr 12 03:51:38 user nova-compute[72081]:   </metadata>
Apr 12 03:51:38 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:51:38 user nova-compute[72081]:     <system>
Apr 12 03:51:38 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:51:38 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:51:38 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:51:38 user nova-compute[72081]:       <entry name="serial">65ba7373-1427-409d-9aa7-6cb7894d665b</entry>
Apr 12 03:51:38 user nova-compute[72081]:       <entry name="uuid">65ba7373-1427-409d-9aa7-6cb7894d665b</entry>
Apr 12 03:51:38 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:51:38 user nova-compute[72081]:     </system>
Apr 12 03:51:38 user nova-compute[72081]:   </sysinfo>
Apr 12 03:51:38 user nova-compute[72081]:   <os>
Apr 12 03:51:38 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:51:38 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:51:38 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:51:38 user nova-compute[72081]:   </os>
Apr 12 03:51:38 user nova-compute[72081]:   <features>
Apr 12 03:51:38 user nova-compute[72081]:     <acpi/>
Apr 12 03:51:38 user nova-compute[72081]:     <apic/>
Apr 12 03:51:38 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:51:38 user nova-compute[72081]:   </features>
Apr 12 03:51:38 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:51:38 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:51:38 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:51:38 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:51:38 user nova-compute[72081]:   </clock>
Apr 12 03:51:38 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:51:38 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:51:38 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:51:38 user nova-compute[72081]:   </cpu>
Apr 12 03:51:38 user nova-compute[72081]:   <devices>
Apr 12 03:51:38 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:51:38 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:51:38 user nova-compute[72081]:       <source dev="/dev/sdb"/>
Apr 12 03:51:38 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:51:38 user nova-compute[72081]:       <serial>f699cce1-7b7a-47fc-9c5e-161e74d0becd</serial>
Apr 12 03:51:38 user nova-compute[72081]:     </disk>
Apr 12 03:51:38 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:51:38 user nova-compute[72081]:       <mac address="fa:16:3e:e9:20:ca"/>
Apr 12 03:51:38 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:51:38 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:51:38 user nova-compute[72081]:       <target dev="tapd7e82a6e-18"/>
Apr 12 03:51:38 user nova-compute[72081]:     </interface>
Apr 12 03:51:38 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:51:38 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/65ba7373-1427-409d-9aa7-6cb7894d665b/console.log" append="off"/>
Apr 12 03:51:38 user nova-compute[72081]:     </serial>
Apr 12 03:51:38 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:51:38 user nova-compute[72081]:     <video>
Apr 12 03:51:38 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:51:38 user nova-compute[72081]:     </video>
Apr 12 03:51:38 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:51:38 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:51:38 user nova-compute[72081]:     </rng>
Apr 12 03:51:38 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:51:38 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:51:38 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:51:38 user nova-compute[72081]:     </memballoon>
Apr 12 03:51:38 user nova-compute[72081]:   </devices>
Apr 12 03:51:38 user nova-compute[72081]: </domain>
Apr 12 03:51:38 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:51:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:49:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-441149863',display_name='tempest-ServerActionsTestJSON-server-441149863',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(12),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-441149863',id=40,image_ref='',info_cache=InstanceInfoCache,instance_type_id=12,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKmrI9wy1CUIwhlKuXvzk0mD7Pt5nZFvI1CsqS0do6Lk76JQOvGl5lvx9rkz7UAM9WGocGXyWSOUOuKTtYZRD2HqKUbTN4VPOoliE1yuu+s0lx7vuZu45Mj7wDb2S8Ksyw==',key_name='tempest-keypair-364276420',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:49:44Z,launched_on='user',locked=False,locked_by=None,memory_mb=192,metadata={},migration_context=MigrationContext,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=Flavor(11),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='815b65aa5ad9442db0a087c646f96bbd',ramdisk_id='',reservation_id='r-7tc52gi6',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1898059859',owner_user_name='tempest-ServerActionsTestJSON-1898059859-project-member'},tags=<?>,task_state='resize_finish',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:51:32Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a7f422f1a05a4a92ba387e424850b360',uuid=65ba7373-1427-409d-9aa7-6cb7894d665b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "address": "fa:16:3e:e9:20:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1452412921-network", "vif_mac": "fa:16:3e:e9:20:ca"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e82a6e-18", "ovs_interfaceid": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:51:38 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converting VIF {"id": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "address": "fa:16:3e:e9:20:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-1452412921-network", "vif_mac": "fa:16:3e:e9:20:ca"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e82a6e-18", "ovs_interfaceid": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:51:38 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e9:20:ca,bridge_name='br-int',has_traffic_filtering=True,id=d7e82a6e-18fe-44f2-b866-f0561d1b8209,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd7e82a6e-18') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:51:38 user nova-compute[72081]: DEBUG os_vif [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:e9:20:ca,bridge_name='br-int',has_traffic_filtering=True,id=d7e82a6e-18fe-44f2-b866-f0561d1b8209,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd7e82a6e-18') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:51:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:51:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:51:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapd7e82a6e-18, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:51:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapd7e82a6e-18, col_values=(('external_ids', {'iface-id': 'd7e82a6e-18fe-44f2-b866-f0561d1b8209', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:e9:20:ca', 'vm-uuid': '65ba7373-1427-409d-9aa7-6cb7894d665b'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:51:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:51:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:38 user nova-compute[72081]: INFO os_vif [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:e9:20:ca,bridge_name='br-int',has_traffic_filtering=True,id=d7e82a6e-18fe-44f2-b866-f0561d1b8209,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd7e82a6e-18')
Apr 12 03:51:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:51:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] No VIF found with MAC fa:16:3e:e9:20:ca, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:51:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:40 user nova-compute[72081]: DEBUG nova.compute.manager [req-85565830-0fb6-4b31-a56c-d3d3bf146cc0 req-cd882521-3b80-427d-a4ba-875b186dc81c service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Received event network-vif-plugged-d7e82a6e-18fe-44f2-b866-f0561d1b8209 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:51:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-85565830-0fb6-4b31-a56c-d3d3bf146cc0 req-cd882521-3b80-427d-a4ba-875b186dc81c service nova] Acquiring lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-85565830-0fb6-4b31-a56c-d3d3bf146cc0 req-cd882521-3b80-427d-a4ba-875b186dc81c service nova] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-85565830-0fb6-4b31-a56c-d3d3bf146cc0 req-cd882521-3b80-427d-a4ba-875b186dc81c service nova] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:40 user nova-compute[72081]: DEBUG nova.compute.manager [req-85565830-0fb6-4b31-a56c-d3d3bf146cc0 req-cd882521-3b80-427d-a4ba-875b186dc81c service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] No waiting events found dispatching network-vif-plugged-d7e82a6e-18fe-44f2-b866-f0561d1b8209 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:51:40 user nova-compute[72081]: WARNING nova.compute.manager [req-85565830-0fb6-4b31-a56c-d3d3bf146cc0 req-cd882521-3b80-427d-a4ba-875b186dc81c service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Received unexpected event network-vif-plugged-d7e82a6e-18fe-44f2-b866-f0561d1b8209 for instance with vm_state active and task_state resize_finish.
Apr 12 03:51:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-685cf456-4a51-4fca-979e-1789f96853bf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "93a2f40a-1eae-4986-b418-ddbf45723196" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-685cf456-4a51-4fca-979e-1789f96853bf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "93a2f40a-1eae-4986-b418-ddbf45723196" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-685cf456-4a51-4fca-979e-1789f96853bf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "93a2f40a-1eae-4986-b418-ddbf45723196-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-685cf456-4a51-4fca-979e-1789f96853bf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "93a2f40a-1eae-4986-b418-ddbf45723196-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-685cf456-4a51-4fca-979e-1789f96853bf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "93a2f40a-1eae-4986-b418-ddbf45723196-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:41 user nova-compute[72081]: INFO nova.compute.manager [None req-685cf456-4a51-4fca-979e-1789f96853bf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Terminating instance
Apr 12 03:51:41 user nova-compute[72081]: DEBUG nova.compute.manager [None req-685cf456-4a51-4fca-979e-1789f96853bf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:51:41 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:51:41 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Removed pending event for 65ba7373-1427-409d-9aa7-6cb7894d665b due to event {{(pid=72081) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 12 03:51:41 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271501.857897, 65ba7373-1427-409d-9aa7-6cb7894d665b => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:51:41 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] VM Resumed (Lifecycle Event)
Apr 12 03:51:41 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Instance running successfully.
Apr 12 03:51:41 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Failed to set time: agent not configured {{(pid=72081) sync_guest_time /opt/stack/nova/nova/virt/libvirt/guest.py:200}}
Apr 12 03:51:41 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1ee91188-0132-4d09-8fdc-1cd4eb0de709 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] finish_migration finished successfully. {{(pid=72081) finish_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11742}}
Apr 12 03:51:41 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:51:41 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: resize_finish, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:51:41 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] During sync_power_state the instance has a pending task (resize_finish). Skip.
Apr 12 03:51:41 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271501.8600814, 65ba7373-1427-409d-9aa7-6cb7894d665b => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:51:41 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] VM Started (Lifecycle Event)
Apr 12 03:51:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:41 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:51:41 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: resize_finish, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:51:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:41 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] During sync_power_state the instance has a pending task (resize_finish). Skip.
Apr 12 03:51:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:42 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681271487.0128117, a6d7d43b-ac65-4ebf-9645-b17e61397076 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:51:42 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] VM Stopped (Lifecycle Event)
Apr 12 03:51:42 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f26d7e4c-1676-4b93-9b69-10211eccef73 None None] [instance: a6d7d43b-ac65-4ebf-9645-b17e61397076] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:51:42 user nova-compute[72081]: DEBUG nova.compute.manager [req-f1c4f97c-d352-4abe-9995-76e66c6b8118 req-ecc68321-6c36-4052-96f4-69077868fa0c service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Received event network-vif-plugged-d7e82a6e-18fe-44f2-b866-f0561d1b8209 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:51:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f1c4f97c-d352-4abe-9995-76e66c6b8118 req-ecc68321-6c36-4052-96f4-69077868fa0c service nova] Acquiring lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f1c4f97c-d352-4abe-9995-76e66c6b8118 req-ecc68321-6c36-4052-96f4-69077868fa0c service nova] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f1c4f97c-d352-4abe-9995-76e66c6b8118 req-ecc68321-6c36-4052-96f4-69077868fa0c service nova] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:42 user nova-compute[72081]: DEBUG nova.compute.manager [req-f1c4f97c-d352-4abe-9995-76e66c6b8118 req-ecc68321-6c36-4052-96f4-69077868fa0c service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] No waiting events found dispatching network-vif-plugged-d7e82a6e-18fe-44f2-b866-f0561d1b8209 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:51:42 user nova-compute[72081]: WARNING nova.compute.manager [req-f1c4f97c-d352-4abe-9995-76e66c6b8118 req-ecc68321-6c36-4052-96f4-69077868fa0c service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Received unexpected event network-vif-plugged-d7e82a6e-18fe-44f2-b866-f0561d1b8209 for instance with vm_state active and task_state resize_finish.
Apr 12 03:51:42 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Instance destroyed successfully.
Apr 12 03:51:42 user nova-compute[72081]: DEBUG nova.objects.instance [None req-685cf456-4a51-4fca-979e-1789f96853bf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lazy-loading 'resources' on Instance uuid 93a2f40a-1eae-4986-b418-ddbf45723196 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:51:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-685cf456-4a51-4fca-979e-1789f96853bf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:48:51Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-127420323',display_name='tempest-AttachVolumeNegativeTest-server-127420323',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-127420323',id=39,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFMWaRzq0i18wkyi3ji2eDJZY0v9caN/UqoTYCms3zb7sFYi3nVI2quTKtNRnFhKWj2bjy1/Mkv/ZsXUW1QOTCPWsEbRbmvA2ipXmyTgXPzbmkqrBL4q39n/p2cfT7w3RA==',key_name='tempest-keypair-422441298',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:48:58Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='d13546149a4e44e1b782b99e9eb452e8',ramdisk_id='',reservation_id='r-om37s4hv',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeNegativeTest-494105310',owner_user_name='tempest-AttachVolumeNegativeTest-494105310-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:48:58Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d52637cca2114e5e99e688ee371c835a',uuid=93a2f40a-1eae-4986-b418-ddbf45723196,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "7afd2456-5a12-4283-8760-46d4c7fe8ad1", "address": "fa:16:3e:ba:1d:ce", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.120", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7afd2456-5a", "ovs_interfaceid": "7afd2456-5a12-4283-8760-46d4c7fe8ad1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:51:42 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-685cf456-4a51-4fca-979e-1789f96853bf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Converting VIF {"id": "7afd2456-5a12-4283-8760-46d4c7fe8ad1", "address": "fa:16:3e:ba:1d:ce", "network": {"id": "ddbce990-4cfa-40bd-b1e1-fc1531789074", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1448596166-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.120", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d13546149a4e44e1b782b99e9eb452e8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7afd2456-5a", "ovs_interfaceid": "7afd2456-5a12-4283-8760-46d4c7fe8ad1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:51:42 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-685cf456-4a51-4fca-979e-1789f96853bf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:ba:1d:ce,bridge_name='br-int',has_traffic_filtering=True,id=7afd2456-5a12-4283-8760-46d4c7fe8ad1,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7afd2456-5a') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:51:42 user nova-compute[72081]: DEBUG os_vif [None req-685cf456-4a51-4fca-979e-1789f96853bf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:ba:1d:ce,bridge_name='br-int',has_traffic_filtering=True,id=7afd2456-5a12-4283-8760-46d4c7fe8ad1,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7afd2456-5a') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:51:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap7afd2456-5a, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:51:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:51:42 user nova-compute[72081]: INFO os_vif [None req-685cf456-4a51-4fca-979e-1789f96853bf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:ba:1d:ce,bridge_name='br-int',has_traffic_filtering=True,id=7afd2456-5a12-4283-8760-46d4c7fe8ad1,network=Network(ddbce990-4cfa-40bd-b1e1-fc1531789074),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7afd2456-5a')
Apr 12 03:51:42 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-685cf456-4a51-4fca-979e-1789f96853bf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Deleting instance files /opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196_del
Apr 12 03:51:42 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-685cf456-4a51-4fca-979e-1789f96853bf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Deletion of /opt/stack/data/nova/instances/93a2f40a-1eae-4986-b418-ddbf45723196_del complete
Apr 12 03:51:42 user nova-compute[72081]: INFO nova.compute.manager [None req-685cf456-4a51-4fca-979e-1789f96853bf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Took 0.88 seconds to destroy the instance on the hypervisor.
Apr 12 03:51:42 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-685cf456-4a51-4fca-979e-1789f96853bf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:51:42 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:51:42 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:51:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:43 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:51:43 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Took 0.84 seconds to deallocate network for instance.
Apr 12 03:51:43 user nova-compute[72081]: DEBUG nova.compute.manager [req-cd636a47-c68b-4a82-a830-58b197a4194f req-8b05ca88-352b-43d8-8b36-931ca1090c8a service nova] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Received event network-vif-deleted-7afd2456-5a12-4283-8760-46d4c7fe8ad1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:51:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-685cf456-4a51-4fca-979e-1789f96853bf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-685cf456-4a51-4fca-979e-1789f96853bf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:43 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-685cf456-4a51-4fca-979e-1789f96853bf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:51:43 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-685cf456-4a51-4fca-979e-1789f96853bf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:51:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-685cf456-4a51-4fca-979e-1789f96853bf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.279s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:43 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-685cf456-4a51-4fca-979e-1789f96853bf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Deleted allocations for instance 93a2f40a-1eae-4986-b418-ddbf45723196
Apr 12 03:51:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-685cf456-4a51-4fca-979e-1789f96853bf tempest-AttachVolumeNegativeTest-494105310 tempest-AttachVolumeNegativeTest-494105310-project-member] Lock "93a2f40a-1eae-4986-b418-ddbf45723196" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.199s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:44 user nova-compute[72081]: DEBUG nova.compute.manager [req-6ea8a2ff-7add-44de-a588-0cf0cdd15b19 req-8f83f3fd-af5e-41ac-bed7-8be05f7f9fa3 service nova] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Received event network-vif-unplugged-7afd2456-5a12-4283-8760-46d4c7fe8ad1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:51:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6ea8a2ff-7add-44de-a588-0cf0cdd15b19 req-8f83f3fd-af5e-41ac-bed7-8be05f7f9fa3 service nova] Acquiring lock "93a2f40a-1eae-4986-b418-ddbf45723196-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6ea8a2ff-7add-44de-a588-0cf0cdd15b19 req-8f83f3fd-af5e-41ac-bed7-8be05f7f9fa3 service nova] Lock "93a2f40a-1eae-4986-b418-ddbf45723196-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6ea8a2ff-7add-44de-a588-0cf0cdd15b19 req-8f83f3fd-af5e-41ac-bed7-8be05f7f9fa3 service nova] Lock "93a2f40a-1eae-4986-b418-ddbf45723196-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:44 user nova-compute[72081]: DEBUG nova.compute.manager [req-6ea8a2ff-7add-44de-a588-0cf0cdd15b19 req-8f83f3fd-af5e-41ac-bed7-8be05f7f9fa3 service nova] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] No waiting events found dispatching network-vif-unplugged-7afd2456-5a12-4283-8760-46d4c7fe8ad1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:51:44 user nova-compute[72081]: WARNING nova.compute.manager [req-6ea8a2ff-7add-44de-a588-0cf0cdd15b19 req-8f83f3fd-af5e-41ac-bed7-8be05f7f9fa3 service nova] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Received unexpected event network-vif-unplugged-7afd2456-5a12-4283-8760-46d4c7fe8ad1 for instance with vm_state deleted and task_state None.
Apr 12 03:51:44 user nova-compute[72081]: DEBUG nova.compute.manager [req-6ea8a2ff-7add-44de-a588-0cf0cdd15b19 req-8f83f3fd-af5e-41ac-bed7-8be05f7f9fa3 service nova] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Received event network-vif-plugged-7afd2456-5a12-4283-8760-46d4c7fe8ad1 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:51:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6ea8a2ff-7add-44de-a588-0cf0cdd15b19 req-8f83f3fd-af5e-41ac-bed7-8be05f7f9fa3 service nova] Acquiring lock "93a2f40a-1eae-4986-b418-ddbf45723196-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:51:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6ea8a2ff-7add-44de-a588-0cf0cdd15b19 req-8f83f3fd-af5e-41ac-bed7-8be05f7f9fa3 service nova] Lock "93a2f40a-1eae-4986-b418-ddbf45723196-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:51:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6ea8a2ff-7add-44de-a588-0cf0cdd15b19 req-8f83f3fd-af5e-41ac-bed7-8be05f7f9fa3 service nova] Lock "93a2f40a-1eae-4986-b418-ddbf45723196-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:51:44 user nova-compute[72081]: DEBUG nova.compute.manager [req-6ea8a2ff-7add-44de-a588-0cf0cdd15b19 req-8f83f3fd-af5e-41ac-bed7-8be05f7f9fa3 service nova] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] No waiting events found dispatching network-vif-plugged-7afd2456-5a12-4283-8760-46d4c7fe8ad1 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:51:44 user nova-compute[72081]: WARNING nova.compute.manager [req-6ea8a2ff-7add-44de-a588-0cf0cdd15b19 req-8f83f3fd-af5e-41ac-bed7-8be05f7f9fa3 service nova] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Received unexpected event network-vif-plugged-7afd2456-5a12-4283-8760-46d4c7fe8ad1 for instance with vm_state deleted and task_state None.
Apr 12 03:51:45 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681271490.2707314, 606425bb-8968-41f9-b3a5-80b6fd188a67 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:51:45 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] VM Stopped (Lifecycle Event)
Apr 12 03:51:45 user nova-compute[72081]: DEBUG nova.compute.manager [None req-c27aefd1-e28c-4645-90f0-46fa29105e57 None None] [instance: 606425bb-8968-41f9-b3a5-80b6fd188a67] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:51:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:57 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681271502.610831, 93a2f40a-1eae-4986-b418-ddbf45723196 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:51:57 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] VM Stopped (Lifecycle Event)
Apr 12 03:51:57 user nova-compute[72081]: DEBUG nova.compute.manager [None req-feafed63-0f36-4afb-8bcc-9c66a59de755 None None] [instance: 93a2f40a-1eae-4986-b418-ddbf45723196] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:51:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:51:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:16 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:52:17 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:52:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:20 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:52:20 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:52:20 user nova-compute[72081]: DEBUG nova.compute.manager [req-f572d2b0-12ea-4460-9a4e-1d836086945d req-bd9815ab-1a91-496d-865d-ca4bbfd66492 service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Received event network-changed-c2ac7de8-155e-4d88-8534-5a5291bd18dc {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:52:20 user nova-compute[72081]: DEBUG nova.compute.manager [req-f572d2b0-12ea-4460-9a4e-1d836086945d req-bd9815ab-1a91-496d-865d-ca4bbfd66492 service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Refreshing instance network info cache due to event network-changed-c2ac7de8-155e-4d88-8534-5a5291bd18dc. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:52:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f572d2b0-12ea-4460-9a4e-1d836086945d req-bd9815ab-1a91-496d-865d-ca4bbfd66492 service nova] Acquiring lock "refresh_cache-7c2344ea-3ed4-40f5-b318-6fa8588b1fed" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:52:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f572d2b0-12ea-4460-9a4e-1d836086945d req-bd9815ab-1a91-496d-865d-ca4bbfd66492 service nova] Acquired lock "refresh_cache-7c2344ea-3ed4-40f5-b318-6fa8588b1fed" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:52:20 user nova-compute[72081]: DEBUG nova.network.neutron [req-f572d2b0-12ea-4460-9a4e-1d836086945d req-bd9815ab-1a91-496d-865d-ca4bbfd66492 service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Refreshing network info cache for port c2ac7de8-155e-4d88-8534-5a5291bd18dc {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:52:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dbc16410-2bf7-4d13-b293-ff8f3a52be06 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:52:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dbc16410-2bf7-4d13-b293-ff8f3a52be06 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:52:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dbc16410-2bf7-4d13-b293-ff8f3a52be06 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:52:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dbc16410-2bf7-4d13-b293-ff8f3a52be06 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:52:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dbc16410-2bf7-4d13-b293-ff8f3a52be06 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:52:20 user nova-compute[72081]: INFO nova.compute.manager [None req-dbc16410-2bf7-4d13-b293-ff8f3a52be06 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Terminating instance
Apr 12 03:52:20 user nova-compute[72081]: DEBUG nova.compute.manager [None req-dbc16410-2bf7-4d13-b293-ff8f3a52be06 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:52:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG nova.compute.manager [req-efc1bf56-efae-4bc4-9f87-38060428bfd7 req-eb26d342-4fdf-4ae4-a90e-5922c13335de service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Received event network-vif-unplugged-c2ac7de8-155e-4d88-8534-5a5291bd18dc {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-efc1bf56-efae-4bc4-9f87-38060428bfd7 req-eb26d342-4fdf-4ae4-a90e-5922c13335de service nova] Acquiring lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-efc1bf56-efae-4bc4-9f87-38060428bfd7 req-eb26d342-4fdf-4ae4-a90e-5922c13335de service nova] Lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-efc1bf56-efae-4bc4-9f87-38060428bfd7 req-eb26d342-4fdf-4ae4-a90e-5922c13335de service nova] Lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG nova.compute.manager [req-efc1bf56-efae-4bc4-9f87-38060428bfd7 req-eb26d342-4fdf-4ae4-a90e-5922c13335de service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] No waiting events found dispatching network-vif-unplugged-c2ac7de8-155e-4d88-8534-5a5291bd18dc {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG nova.compute.manager [req-efc1bf56-efae-4bc4-9f87-38060428bfd7 req-eb26d342-4fdf-4ae4-a90e-5922c13335de service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Received event network-vif-unplugged-c2ac7de8-155e-4d88-8534-5a5291bd18dc for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:52:21 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Instance destroyed successfully.
Apr 12 03:52:21 user nova-compute[72081]: DEBUG nova.objects.instance [None req-dbc16410-2bf7-4d13-b293-ff8f3a52be06 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lazy-loading 'resources' on Instance uuid 7c2344ea-3ed4-40f5-b318-6fa8588b1fed {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-dbc16410-2bf7-4d13-b293-ff8f3a52be06 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:45:31Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestStampPattern-server-1815563360',display_name='tempest-TestStampPattern-server-1815563360',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-teststamppattern-server-1815563360',id=34,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJxviRdZviloE/Y8vy1de4Nl8WuqpnNhf4wG8IpFYt8u0P1qgKwJzMzVj8wiWWgzOhRDflg4OM3oRDBzEH9LU9bTdJCvVUrt7G9IZqwugvdWPxD2R1lvN0sxiTneX8pngQ==',key_name='tempest-TestStampPattern-474439147',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:45:38Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='c426e52304c14ed9b0c1aed69691bc3d',ramdisk_id='',reservation_id='r-5vht163x',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestStampPattern-707517070',owner_user_name='tempest-TestStampPattern-707517070-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:48:30Z,user_data=None,user_id='88c496d8e488434e833e67df25def576',uuid=7c2344ea-3ed4-40f5-b318-6fa8588b1fed,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "address": "fa:16:3e:6d:2b:fe", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.98", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2ac7de8-15", "ovs_interfaceid": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-dbc16410-2bf7-4d13-b293-ff8f3a52be06 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Converting VIF {"id": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "address": "fa:16:3e:6d:2b:fe", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.98", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2ac7de8-15", "ovs_interfaceid": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-dbc16410-2bf7-4d13-b293-ff8f3a52be06 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:6d:2b:fe,bridge_name='br-int',has_traffic_filtering=True,id=c2ac7de8-155e-4d88-8534-5a5291bd18dc,network=Network(8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc2ac7de8-15') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG os_vif [None req-dbc16410-2bf7-4d13-b293-ff8f3a52be06 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:6d:2b:fe,bridge_name='br-int',has_traffic_filtering=True,id=c2ac7de8-155e-4d88-8534-5a5291bd18dc,network=Network(8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc2ac7de8-15') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapc2ac7de8-15, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:52:21 user nova-compute[72081]: INFO os_vif [None req-dbc16410-2bf7-4d13-b293-ff8f3a52be06 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:6d:2b:fe,bridge_name='br-int',has_traffic_filtering=True,id=c2ac7de8-155e-4d88-8534-5a5291bd18dc,network=Network(8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc2ac7de8-15')
Apr 12 03:52:21 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-dbc16410-2bf7-4d13-b293-ff8f3a52be06 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Deleting instance files /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed_del
Apr 12 03:52:21 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-dbc16410-2bf7-4d13-b293-ff8f3a52be06 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Deletion of /opt/stack/data/nova/instances/7c2344ea-3ed4-40f5-b318-6fa8588b1fed_del complete
Apr 12 03:52:21 user nova-compute[72081]: INFO nova.compute.manager [None req-dbc16410-2bf7-4d13-b293-ff8f3a52be06 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Took 0.87 seconds to destroy the instance on the hypervisor.
Apr 12 03:52:21 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-dbc16410-2bf7-4d13-b293-ff8f3a52be06 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:52:21 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Error from libvirt while getting description of instance-00000022: [Error Code 42] Domain not found: no domain with matching uuid '7c2344ea-3ed4-40f5-b318-6fa8588b1fed' (instance-00000022): libvirt.libvirtError: Domain not found: no domain with matching uuid '7c2344ea-3ed4-40f5-b318-6fa8588b1fed' (instance-00000022)
Apr 12 03:52:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:52:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json" returned: 0 in 0.151s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:52:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:52:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:52:22 user nova-compute[72081]: DEBUG nova.network.neutron [req-f572d2b0-12ea-4460-9a4e-1d836086945d req-bd9815ab-1a91-496d-865d-ca4bbfd66492 service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Updated VIF entry in instance network info cache for port c2ac7de8-155e-4d88-8534-5a5291bd18dc. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:52:22 user nova-compute[72081]: DEBUG nova.network.neutron [req-f572d2b0-12ea-4460-9a4e-1d836086945d req-bd9815ab-1a91-496d-865d-ca4bbfd66492 service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Updating instance_info_cache with network_info: [{"id": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "address": "fa:16:3e:6d:2b:fe", "network": {"id": "8ef5a542-2a86-44d7-9ec3-dbce4cd3af6a", "bridge": "br-int", "label": "tempest-TestStampPattern-798327415-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c426e52304c14ed9b0c1aed69691bc3d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2ac7de8-15", "ovs_interfaceid": "c2ac7de8-155e-4d88-8534-5a5291bd18dc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:52:22 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:52:22 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:52:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8495MB free_disk=26.04925537109375GB free_vcpus=10 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:52:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:52:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:52:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f572d2b0-12ea-4460-9a4e-1d836086945d req-bd9815ab-1a91-496d-865d-ca4bbfd66492 service nova] Releasing lock "refresh_cache-7c2344ea-3ed4-40f5-b318-6fa8588b1fed" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:52:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Applying migration context for instance 65ba7373-1427-409d-9aa7-6cb7894d665b as it has an incoming, in-progress migration 08dbfa70-c3f3-44ff-8dd8-3ea443fb7244. Migration status is finished {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:950}}
Apr 12 03:52:22 user nova-compute[72081]: INFO nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Updating resource usage from migration 08dbfa70-c3f3-44ff-8dd8-3ea443fb7244
Apr 12 03:52:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 7c2344ea-3ed4-40f5-b318-6fa8588b1fed actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:52:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:52:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Migration 08dbfa70-c3f3-44ff-8dd8-3ea443fb7244 is active on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 128}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1637}}
Apr 12 03:52:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 65ba7373-1427-409d-9aa7-6cb7894d665b actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 192, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:52:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 4 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:52:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=1088MB phys_disk=40GB used_disk=2GB total_vcpus=12 used_vcpus=4 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:52:22 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:52:22 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Took 1.11 seconds to deallocate network for instance.
Apr 12 03:52:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dbc16410-2bf7-4d13-b293-ff8f3a52be06 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:52:22 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.454s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dbc16410-2bf7-4d13-b293-ff8f3a52be06 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.091s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-dbc16410-2bf7-4d13-b293-ff8f3a52be06 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-dbc16410-2bf7-4d13-b293-ff8f3a52be06 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dbc16410-2bf7-4d13-b293-ff8f3a52be06 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.217s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "9a705900-a0de-46e3-9af4-d07911f181c0" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "9a705900-a0de-46e3-9af4-d07911f181c0" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:52:23 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-dbc16410-2bf7-4d13-b293-ff8f3a52be06 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Deleted allocations for instance 7c2344ea-3ed4-40f5-b318-6fa8588b1fed
Apr 12 03:52:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-dbc16410-2bf7-4d13-b293-ff8f3a52be06 tempest-TestStampPattern-707517070 tempest-TestStampPattern-707517070-project-member] Lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.488s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:52:23 user nova-compute[72081]: INFO nova.compute.claims [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Claim successful on node user
Apr 12 03:52:23 user nova-compute[72081]: DEBUG nova.compute.manager [req-7ff5b304-32c3-4007-8f0e-b457dc6ff050 req-839d9390-9db2-4c44-bc9d-5e6f2fc044b8 service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Received event network-vif-plugged-c2ac7de8-155e-4d88-8534-5a5291bd18dc {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7ff5b304-32c3-4007-8f0e-b457dc6ff050 req-839d9390-9db2-4c44-bc9d-5e6f2fc044b8 service nova] Acquiring lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7ff5b304-32c3-4007-8f0e-b457dc6ff050 req-839d9390-9db2-4c44-bc9d-5e6f2fc044b8 service nova] Lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7ff5b304-32c3-4007-8f0e-b457dc6ff050 req-839d9390-9db2-4c44-bc9d-5e6f2fc044b8 service nova] Lock "7c2344ea-3ed4-40f5-b318-6fa8588b1fed-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG nova.compute.manager [req-7ff5b304-32c3-4007-8f0e-b457dc6ff050 req-839d9390-9db2-4c44-bc9d-5e6f2fc044b8 service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] No waiting events found dispatching network-vif-plugged-c2ac7de8-155e-4d88-8534-5a5291bd18dc {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:52:23 user nova-compute[72081]: WARNING nova.compute.manager [req-7ff5b304-32c3-4007-8f0e-b457dc6ff050 req-839d9390-9db2-4c44-bc9d-5e6f2fc044b8 service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Received unexpected event network-vif-plugged-c2ac7de8-155e-4d88-8534-5a5291bd18dc for instance with vm_state deleted and task_state None.
Apr 12 03:52:23 user nova-compute[72081]: DEBUG nova.compute.manager [req-7ff5b304-32c3-4007-8f0e-b457dc6ff050 req-839d9390-9db2-4c44-bc9d-5e6f2fc044b8 service nova] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Received event network-vif-deleted-c2ac7de8-155e-4d88-8534-5a5291bd18dc {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.256s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:52:23 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:52:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG nova.policy [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '872fbc3d59274cd480c17b4b7342499a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a341e582192a47548df8627d82c96ea8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:52:23 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Creating image(s)
Apr 12 03:52:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "/opt/stack/data/nova/instances/9a705900-a0de-46e3-9af4-d07911f181c0/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "/opt/stack/data/nova/instances/9a705900-a0de-46e3-9af4-d07911f181c0/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "/opt/stack/data/nova/instances/9a705900-a0de-46e3-9af4-d07911f181c0/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.006s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:52:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.195s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.179s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/9a705900-a0de-46e3-9af4-d07911f181c0/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/9a705900-a0de-46e3-9af4-d07911f181c0/disk 1073741824" returned: 0 in 0.067s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.253s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.179s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Checking if we can resize image /opt/stack/data/nova/instances/9a705900-a0de-46e3-9af4-d07911f181c0/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9a705900-a0de-46e3-9af4-d07911f181c0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9a705900-a0de-46e3-9af4-d07911f181c0/disk --force-share --output=json" returned: 0 in 0.200s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Cannot resize image /opt/stack/data/nova/instances/9a705900-a0de-46e3-9af4-d07911f181c0/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG nova.objects.instance [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'migration_context' on Instance uuid 9a705900-a0de-46e3-9af4-d07911f181c0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Ensure instance console log exists: /opt/stack/data/nova/instances/9a705900-a0de-46e3-9af4-d07911f181c0/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:52:24 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Successfully created port: 55b4c29c-432e-4323-a5d6-fc62a42f8655 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:52:25 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Successfully updated port: 55b4c29c-432e-4323-a5d6-fc62a42f8655 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:52:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "refresh_cache-9a705900-a0de-46e3-9af4-d07911f181c0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:52:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquired lock "refresh_cache-9a705900-a0de-46e3-9af4-d07911f181c0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:52:25 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:52:25 user nova-compute[72081]: DEBUG nova.compute.manager [req-6886d3ae-5f34-4010-833e-64f461485936 req-fb30a602-03ef-4952-a85e-7abc4e71f591 service nova] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Received event network-changed-55b4c29c-432e-4323-a5d6-fc62a42f8655 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:52:25 user nova-compute[72081]: DEBUG nova.compute.manager [req-6886d3ae-5f34-4010-833e-64f461485936 req-fb30a602-03ef-4952-a85e-7abc4e71f591 service nova] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Refreshing instance network info cache due to event network-changed-55b4c29c-432e-4323-a5d6-fc62a42f8655. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:52:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6886d3ae-5f34-4010-833e-64f461485936 req-fb30a602-03ef-4952-a85e-7abc4e71f591 service nova] Acquiring lock "refresh_cache-9a705900-a0de-46e3-9af4-d07911f181c0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:52:25 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.network.neutron [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Updating instance_info_cache with network_info: [{"id": "55b4c29c-432e-4323-a5d6-fc62a42f8655", "address": "fa:16:3e:e8:16:de", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55b4c29c-43", "ovs_interfaceid": "55b4c29c-432e-4323-a5d6-fc62a42f8655", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Releasing lock "refresh_cache-9a705900-a0de-46e3-9af4-d07911f181c0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Instance network_info: |[{"id": "55b4c29c-432e-4323-a5d6-fc62a42f8655", "address": "fa:16:3e:e8:16:de", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55b4c29c-43", "ovs_interfaceid": "55b4c29c-432e-4323-a5d6-fc62a42f8655", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6886d3ae-5f34-4010-833e-64f461485936 req-fb30a602-03ef-4952-a85e-7abc4e71f591 service nova] Acquired lock "refresh_cache-9a705900-a0de-46e3-9af4-d07911f181c0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.network.neutron [req-6886d3ae-5f34-4010-833e-64f461485936 req-fb30a602-03ef-4952-a85e-7abc4e71f591 service nova] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Refreshing network info cache for port 55b4c29c-432e-4323-a5d6-fc62a42f8655 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Start _get_guest_xml network_info=[{"id": "55b4c29c-432e-4323-a5d6-fc62a42f8655", "address": "fa:16:3e:e8:16:de", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55b4c29c-43", "ovs_interfaceid": "55b4c29c-432e-4323-a5d6-fc62a42f8655", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:52:26 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:52:26 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:52:22Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1419019551',display_name='tempest-AttachVolumeTestJSON-server-1419019551',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1419019551',id=41,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBF5dfRZfMAh+PjjlIpXMM5gTNQ/AVHmZ6QANnWldIoSsGeWR4SRx5i4ef7EgAPnNzMv6qe56Q5J7su4WXqBoiAy/S0b9fCrjSIUZQ+BsTnQye+GfEYaIPsBJDEmU+yv9Hw==',key_name='tempest-keypair-1939295877',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a341e582192a47548df8627d82c96ea8',ramdisk_id='',reservation_id='r-upbjpoau',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeTestJSON-617614045',owner_user_name='tempest-AttachVolumeTestJSON-617614045-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:52:24Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='872fbc3d59274cd480c17b4b7342499a',uuid=9a705900-a0de-46e3-9af4-d07911f181c0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "55b4c29c-432e-4323-a5d6-fc62a42f8655", "address": "fa:16:3e:e8:16:de", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55b4c29c-43", "ovs_interfaceid": "55b4c29c-432e-4323-a5d6-fc62a42f8655", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Converting VIF {"id": "55b4c29c-432e-4323-a5d6-fc62a42f8655", "address": "fa:16:3e:e8:16:de", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55b4c29c-43", "ovs_interfaceid": "55b4c29c-432e-4323-a5d6-fc62a42f8655", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e8:16:de,bridge_name='br-int',has_traffic_filtering=True,id=55b4c29c-432e-4323-a5d6-fc62a42f8655,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap55b4c29c-43') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.objects.instance [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'pci_devices' on Instance uuid 9a705900-a0de-46e3-9af4-d07911f181c0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:52:26 user nova-compute[72081]:   <uuid>9a705900-a0de-46e3-9af4-d07911f181c0</uuid>
Apr 12 03:52:26 user nova-compute[72081]:   <name>instance-00000029</name>
Apr 12 03:52:26 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:52:26 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:52:26 user nova-compute[72081]:   <metadata>
Apr 12 03:52:26 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:52:26 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:52:26 user nova-compute[72081]:       <nova:name>tempest-AttachVolumeTestJSON-server-1419019551</nova:name>
Apr 12 03:52:26 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:52:26</nova:creationTime>
Apr 12 03:52:26 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:52:26 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:52:26 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:52:26 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:52:26 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:52:26 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:52:26 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:52:26 user nova-compute[72081]:       <nova:owner>
Apr 12 03:52:26 user nova-compute[72081]:         <nova:user uuid="872fbc3d59274cd480c17b4b7342499a">tempest-AttachVolumeTestJSON-617614045-project-member</nova:user>
Apr 12 03:52:26 user nova-compute[72081]:         <nova:project uuid="a341e582192a47548df8627d82c96ea8">tempest-AttachVolumeTestJSON-617614045</nova:project>
Apr 12 03:52:26 user nova-compute[72081]:       </nova:owner>
Apr 12 03:52:26 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:52:26 user nova-compute[72081]:       <nova:ports>
Apr 12 03:52:26 user nova-compute[72081]:         <nova:port uuid="55b4c29c-432e-4323-a5d6-fc62a42f8655">
Apr 12 03:52:26 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.11" ipVersion="4"/>
Apr 12 03:52:26 user nova-compute[72081]:         </nova:port>
Apr 12 03:52:26 user nova-compute[72081]:       </nova:ports>
Apr 12 03:52:26 user nova-compute[72081]:     </nova:instance>
Apr 12 03:52:26 user nova-compute[72081]:   </metadata>
Apr 12 03:52:26 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:52:26 user nova-compute[72081]:     <system>
Apr 12 03:52:26 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:52:26 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:52:26 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:52:26 user nova-compute[72081]:       <entry name="serial">9a705900-a0de-46e3-9af4-d07911f181c0</entry>
Apr 12 03:52:26 user nova-compute[72081]:       <entry name="uuid">9a705900-a0de-46e3-9af4-d07911f181c0</entry>
Apr 12 03:52:26 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:52:26 user nova-compute[72081]:     </system>
Apr 12 03:52:26 user nova-compute[72081]:   </sysinfo>
Apr 12 03:52:26 user nova-compute[72081]:   <os>
Apr 12 03:52:26 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:52:26 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:52:26 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:52:26 user nova-compute[72081]:   </os>
Apr 12 03:52:26 user nova-compute[72081]:   <features>
Apr 12 03:52:26 user nova-compute[72081]:     <acpi/>
Apr 12 03:52:26 user nova-compute[72081]:     <apic/>
Apr 12 03:52:26 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:52:26 user nova-compute[72081]:   </features>
Apr 12 03:52:26 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:52:26 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:52:26 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:52:26 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:52:26 user nova-compute[72081]:   </clock>
Apr 12 03:52:26 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:52:26 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:52:26 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:52:26 user nova-compute[72081]:   </cpu>
Apr 12 03:52:26 user nova-compute[72081]:   <devices>
Apr 12 03:52:26 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:52:26 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:52:26 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/9a705900-a0de-46e3-9af4-d07911f181c0/disk"/>
Apr 12 03:52:26 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:52:26 user nova-compute[72081]:     </disk>
Apr 12 03:52:26 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:52:26 user nova-compute[72081]:       <mac address="fa:16:3e:e8:16:de"/>
Apr 12 03:52:26 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:52:26 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:52:26 user nova-compute[72081]:       <target dev="tap55b4c29c-43"/>
Apr 12 03:52:26 user nova-compute[72081]:     </interface>
Apr 12 03:52:26 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:52:26 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/9a705900-a0de-46e3-9af4-d07911f181c0/console.log" append="off"/>
Apr 12 03:52:26 user nova-compute[72081]:     </serial>
Apr 12 03:52:26 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:52:26 user nova-compute[72081]:     <video>
Apr 12 03:52:26 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:52:26 user nova-compute[72081]:     </video>
Apr 12 03:52:26 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:52:26 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:52:26 user nova-compute[72081]:     </rng>
Apr 12 03:52:26 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:52:26 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:52:26 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:52:26 user nova-compute[72081]:     </memballoon>
Apr 12 03:52:26 user nova-compute[72081]:   </devices>
Apr 12 03:52:26 user nova-compute[72081]: </domain>
Apr 12 03:52:26 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:52:22Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1419019551',display_name='tempest-AttachVolumeTestJSON-server-1419019551',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1419019551',id=41,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBF5dfRZfMAh+PjjlIpXMM5gTNQ/AVHmZ6QANnWldIoSsGeWR4SRx5i4ef7EgAPnNzMv6qe56Q5J7su4WXqBoiAy/S0b9fCrjSIUZQ+BsTnQye+GfEYaIPsBJDEmU+yv9Hw==',key_name='tempest-keypair-1939295877',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a341e582192a47548df8627d82c96ea8',ramdisk_id='',reservation_id='r-upbjpoau',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeTestJSON-617614045',owner_user_name='tempest-AttachVolumeTestJSON-617614045-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:52:24Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='872fbc3d59274cd480c17b4b7342499a',uuid=9a705900-a0de-46e3-9af4-d07911f181c0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "55b4c29c-432e-4323-a5d6-fc62a42f8655", "address": "fa:16:3e:e8:16:de", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55b4c29c-43", "ovs_interfaceid": "55b4c29c-432e-4323-a5d6-fc62a42f8655", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Converting VIF {"id": "55b4c29c-432e-4323-a5d6-fc62a42f8655", "address": "fa:16:3e:e8:16:de", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55b4c29c-43", "ovs_interfaceid": "55b4c29c-432e-4323-a5d6-fc62a42f8655", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e8:16:de,bridge_name='br-int',has_traffic_filtering=True,id=55b4c29c-432e-4323-a5d6-fc62a42f8655,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap55b4c29c-43') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG os_vif [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:e8:16:de,bridge_name='br-int',has_traffic_filtering=True,id=55b4c29c-432e-4323-a5d6-fc62a42f8655,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap55b4c29c-43') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap55b4c29c-43, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap55b4c29c-43, col_values=(('external_ids', {'iface-id': '55b4c29c-432e-4323-a5d6-fc62a42f8655', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:e8:16:de', 'vm-uuid': '9a705900-a0de-46e3-9af4-d07911f181c0'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:26 user nova-compute[72081]: INFO os_vif [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:e8:16:de,bridge_name='br-int',has_traffic_filtering=True,id=55b4c29c-432e-4323-a5d6-fc62a42f8655,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap55b4c29c-43')
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] No VIF found with MAC fa:16:3e:e8:16:de, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.network.neutron [req-6886d3ae-5f34-4010-833e-64f461485936 req-fb30a602-03ef-4952-a85e-7abc4e71f591 service nova] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Updated VIF entry in instance network info cache for port 55b4c29c-432e-4323-a5d6-fc62a42f8655. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.network.neutron [req-6886d3ae-5f34-4010-833e-64f461485936 req-fb30a602-03ef-4952-a85e-7abc4e71f591 service nova] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Updating instance_info_cache with network_info: [{"id": "55b4c29c-432e-4323-a5d6-fc62a42f8655", "address": "fa:16:3e:e8:16:de", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55b4c29c-43", "ovs_interfaceid": "55b4c29c-432e-4323-a5d6-fc62a42f8655", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6886d3ae-5f34-4010-833e-64f461485936 req-fb30a602-03ef-4952-a85e-7abc4e71f591 service nova] Releasing lock "refresh_cache-9a705900-a0de-46e3-9af4-d07911f181c0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-8d533f5f-dbed-4878-a9b1-2ce4fae8fa13" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-8d533f5f-dbed-4878-a9b1-2ce4fae8fa13" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:52:26 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 03:52:27 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Updating instance_info_cache with network_info: [{"id": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "address": "fa:16:3e:e2:fd:44", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2ef93926-6c", "ovs_interfaceid": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:52:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-8d533f5f-dbed-4878-a9b1-2ce4fae8fa13" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:52:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 03:52:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:27 user nova-compute[72081]: DEBUG nova.compute.manager [req-57b550ba-1062-418e-a2e0-55735c1ab936 req-155c3a10-780d-445f-b805-4cf56f2edba3 service nova] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Received event network-vif-plugged-55b4c29c-432e-4323-a5d6-fc62a42f8655 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:52:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-57b550ba-1062-418e-a2e0-55735c1ab936 req-155c3a10-780d-445f-b805-4cf56f2edba3 service nova] Acquiring lock "9a705900-a0de-46e3-9af4-d07911f181c0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:52:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-57b550ba-1062-418e-a2e0-55735c1ab936 req-155c3a10-780d-445f-b805-4cf56f2edba3 service nova] Lock "9a705900-a0de-46e3-9af4-d07911f181c0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:52:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-57b550ba-1062-418e-a2e0-55735c1ab936 req-155c3a10-780d-445f-b805-4cf56f2edba3 service nova] Lock "9a705900-a0de-46e3-9af4-d07911f181c0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:52:27 user nova-compute[72081]: DEBUG nova.compute.manager [req-57b550ba-1062-418e-a2e0-55735c1ab936 req-155c3a10-780d-445f-b805-4cf56f2edba3 service nova] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] No waiting events found dispatching network-vif-plugged-55b4c29c-432e-4323-a5d6-fc62a42f8655 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:52:27 user nova-compute[72081]: WARNING nova.compute.manager [req-57b550ba-1062-418e-a2e0-55735c1ab936 req-155c3a10-780d-445f-b805-4cf56f2edba3 service nova] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Received unexpected event network-vif-plugged-55b4c29c-432e-4323-a5d6-fc62a42f8655 for instance with vm_state building and task_state spawning.
Apr 12 03:52:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:29 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271549.772794, 9a705900-a0de-46e3-9af4-d07911f181c0 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:52:29 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] VM Resumed (Lifecycle Event)
Apr 12 03:52:29 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:52:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:52:29 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Instance spawned successfully.
Apr 12 03:52:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:52:29 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:52:29 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:52:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:52:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:52:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:52:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:52:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:52:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:52:29 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:52:29 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271549.7740402, 9a705900-a0de-46e3-9af4-d07911f181c0 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:52:29 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] VM Started (Lifecycle Event)
Apr 12 03:52:29 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:52:29 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:52:29 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:52:29 user nova-compute[72081]: INFO nova.compute.manager [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Took 6.02 seconds to spawn the instance on the hypervisor.
Apr 12 03:52:29 user nova-compute[72081]: DEBUG nova.compute.manager [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:52:29 user nova-compute[72081]: INFO nova.compute.manager [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Took 6.61 seconds to build instance.
Apr 12 03:52:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-b3c8527e-44b6-47aa-b80b-f0185dde6575 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "9a705900-a0de-46e3-9af4-d07911f181c0" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.716s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:52:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-07a1af10-4249-47c4-b3f7-81aa0a7ee7f3 req-d910c725-a414-49d4-9ff2-fd5bc4298f54 service nova] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Received event network-vif-plugged-55b4c29c-432e-4323-a5d6-fc62a42f8655 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:52:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-07a1af10-4249-47c4-b3f7-81aa0a7ee7f3 req-d910c725-a414-49d4-9ff2-fd5bc4298f54 service nova] Acquiring lock "9a705900-a0de-46e3-9af4-d07911f181c0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:52:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-07a1af10-4249-47c4-b3f7-81aa0a7ee7f3 req-d910c725-a414-49d4-9ff2-fd5bc4298f54 service nova] Lock "9a705900-a0de-46e3-9af4-d07911f181c0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:52:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-07a1af10-4249-47c4-b3f7-81aa0a7ee7f3 req-d910c725-a414-49d4-9ff2-fd5bc4298f54 service nova] Lock "9a705900-a0de-46e3-9af4-d07911f181c0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:52:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-07a1af10-4249-47c4-b3f7-81aa0a7ee7f3 req-d910c725-a414-49d4-9ff2-fd5bc4298f54 service nova] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] No waiting events found dispatching network-vif-plugged-55b4c29c-432e-4323-a5d6-fc62a42f8655 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:52:30 user nova-compute[72081]: WARNING nova.compute.manager [req-07a1af10-4249-47c4-b3f7-81aa0a7ee7f3 req-d910c725-a414-49d4-9ff2-fd5bc4298f54 service nova] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Received unexpected event network-vif-plugged-55b4c29c-432e-4323-a5d6-fc62a42f8655 for instance with vm_state active and task_state None.
Apr 12 03:52:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:36 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681271541.6629055, 7c2344ea-3ed4-40f5-b318-6fa8588b1fed => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:52:36 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] VM Stopped (Lifecycle Event)
Apr 12 03:52:36 user nova-compute[72081]: DEBUG nova.compute.manager [None req-7c8ec700-f4fb-46ca-961a-f82db2a1ce4b None None] [instance: 7c2344ea-3ed4-40f5-b318-6fa8588b1fed] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:52:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:52:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:52:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:06 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Acquiring lock "1e4dae15-f69f-4e44-959a-7480e185020b" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "1e4dae15-f69f-4e44-959a-7480e185020b" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:53:10 user nova-compute[72081]: INFO nova.compute.claims [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Claim successful on node user
Apr 12 03:53:10 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.282s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:53:10 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:53:10 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:53:10 user nova-compute[72081]: INFO nova.virt.block_device [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Booting with volume d6a07e3a-be2a-4191-b1b7-09667064fb02 at /dev/vda
Apr 12 03:53:10 user nova-compute[72081]: DEBUG nova.policy [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '447437f9d71b4f8ab0da168f0fca732f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd8ec2318ea05403db924630461ea77e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG os_brick.utils [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b8cc3e2d-086b-4a84-ae9a-6d7deaef54c0]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e31ea3ff-3067-44f1-8ef0-93b6a68f8726]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[fa5babe6-a3ea-41ba-9ffe-6d8ee56f891f]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[eb19a488-cf74-473b-a512-9a20420655c5]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:53:10 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:53:10 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3aee8c6d-3421-4901-8486-8b6c6f9ca4b8]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG os_brick.utils [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] <== get_connector_properties: return (128ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:53:10 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Updating existing volume attachment record: 46821723-f94d-4c2c-9a32-5057bfe81335 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:53:11 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Successfully created port: 1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:53:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:12 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Successfully updated port: 1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:53:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Acquiring lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:53:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Acquired lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:53:12 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:53:12 user nova-compute[72081]: DEBUG nova.compute.manager [req-65fea8e4-fdce-4532-a65a-4eefb75dc5ea req-0b4e498f-7c71-41da-9ca4-19eae2424627 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Received event network-changed-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:53:12 user nova-compute[72081]: DEBUG nova.compute.manager [req-65fea8e4-fdce-4532-a65a-4eefb75dc5ea req-0b4e498f-7c71-41da-9ca4-19eae2424627 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Refreshing instance network info cache due to event network-changed-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:53:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-65fea8e4-fdce-4532-a65a-4eefb75dc5ea req-0b4e498f-7c71-41da-9ca4-19eae2424627 service nova] Acquiring lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:53:12 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:53:12 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Updating instance_info_cache with network_info: [{"id": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "address": "fa:16:3e:e9:ff:d5", "network": {"id": "99c0b9a9-e1b4-43d1-bbbb-66560eca330a", "bridge": "br-int", "label": "tempest-TestShelveInstance-2092145606-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d8ec2318ea05403db924630461ea77e5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be56e0c-9e", "ovs_interfaceid": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:53:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Releasing lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:53:12 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Instance network_info: |[{"id": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "address": "fa:16:3e:e9:ff:d5", "network": {"id": "99c0b9a9-e1b4-43d1-bbbb-66560eca330a", "bridge": "br-int", "label": "tempest-TestShelveInstance-2092145606-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d8ec2318ea05403db924630461ea77e5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be56e0c-9e", "ovs_interfaceid": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:53:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-65fea8e4-fdce-4532-a65a-4eefb75dc5ea req-0b4e498f-7c71-41da-9ca4-19eae2424627 service nova] Acquired lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:53:12 user nova-compute[72081]: DEBUG nova.network.neutron [req-65fea8e4-fdce-4532-a65a-4eefb75dc5ea req-0b4e498f-7c71-41da-9ca4-19eae2424627 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Refreshing network info cache for port 1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:53:13 user nova-compute[72081]: DEBUG nova.network.neutron [req-65fea8e4-fdce-4532-a65a-4eefb75dc5ea req-0b4e498f-7c71-41da-9ca4-19eae2424627 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Updated VIF entry in instance network info cache for port 1be56e0c-9ee5-4df6-bbd6-f866a4e54e90. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:53:13 user nova-compute[72081]: DEBUG nova.network.neutron [req-65fea8e4-fdce-4532-a65a-4eefb75dc5ea req-0b4e498f-7c71-41da-9ca4-19eae2424627 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Updating instance_info_cache with network_info: [{"id": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "address": "fa:16:3e:e9:ff:d5", "network": {"id": "99c0b9a9-e1b4-43d1-bbbb-66560eca330a", "bridge": "br-int", "label": "tempest-TestShelveInstance-2092145606-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d8ec2318ea05403db924630461ea77e5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be56e0c-9e", "ovs_interfaceid": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:53:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-65fea8e4-fdce-4532-a65a-4eefb75dc5ea req-0b4e498f-7c71-41da-9ca4-19eae2424627 service nova] Releasing lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:53:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:53:15 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Creating image(s)
Apr 12 03:53:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Did not create local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Ensure instance console log exists: /opt/stack/data/nova/instances/1e4dae15-f69f-4e44-959a-7480e185020b/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Start _get_guest_xml network_info=[{"id": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "address": "fa:16:3e:e9:ff:d5", "network": {"id": "99c0b9a9-e1b4-43d1-bbbb-66560eca330a", "bridge": "br-int", "label": "tempest-TestShelveInstance-2092145606-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d8ec2318ea05403db924630461ea77e5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be56e0c-9e", "ovs_interfaceid": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OZAguYvD', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '1e4dae15-f69f-4e44-959a-7480e185020b', 'attached_at': '', 'detached_at': '', 'volume_id': 'd6a07e3a-be2a-4191-b1b7-09667064fb02', 'serial': 'd6a07e3a-be2a-4191-b1b7-09667064fb02'}, 'disk_bus': 'virtio', 'delete_on_termination': True, 'boot_index': 0, 'guest_format': None, 'device_type': 'disk', 'attachment_id': '46821723-f94d-4c2c-9a32-5057bfe81335', 'volume_type': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:53:15 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:53:15 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:53:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OZAguYvD', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:53:15 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:53:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:53:15 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:53:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:53:15 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:53:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[99e2f893-4e3c-4638-b2fe-1c9f0e80f4ba]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:53:15 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c78375fc-3156-4ed4-8509-a1c5c027817b]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02] added
Apr 12 03:53:15 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5459f628-3674-4a72-9046-e7e1b59edfc1]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ff45d486-59b8-4934-aebb-2487477d874d]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v OZAguYvD {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v OZAguYvD" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0fff385a-0f05-4fb8-a38b-468de1907ed1]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'OZAguYvD'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b1679887-19c5-4eec-9853-c40ef3f730b8]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7eb6ee8a-f303-44fc-87cc-8cfbe11f8b0e]: (4, ('tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:53:15 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsi session list stdout=tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:53:15 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:53:15 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:53:16 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:16 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --login" returned: 0 in 1.052s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:53:16 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[46e44622-240a-4f09-8914-2a35a213d378]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:53:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02, portal: 172.16.0.220,3260]
Apr 12 03:53:16 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02, portal: 172.16.0.220,3260] successful.
Apr 12 03:53:16 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:53:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:53:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:53:16 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5c747380-66fc-4b08-bf0d-9008dd0b123c]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:53:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:53:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:53:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:53:16 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[37040787-03f2-4b97-a837-d24e5727c6e2]: (4, ('tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:53:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:53:16 user nova-compute[72081]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)
Apr 12 03:53:16 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:53:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsi session list stdout=tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:53:16 user nova-compute[72081]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)
Apr 12 03:53:16 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:53:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.270s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:53:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:53:16 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] HCTL ('34', '-', '-', 0) found on session 46 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:53:16 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:53:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:53:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:53:16 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6861b25e-2cf9-4585-9c11-d79d4061e7ca]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:53:16 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Searching for a device in session 46 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:53:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OZAguYvD', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1dd8654d-0753-439b-baa4-fe52888fd4cf tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "65ba7373-1427-409d-9aa7-6cb7894d665b" by "nova.compute.manager.ComputeManager.confirm_resize.<locals>.do_confirm_resize" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1dd8654d-0753-439b-baa4-fe52888fd4cf tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b" acquired by "nova.compute.manager.ComputeManager.confirm_resize.<locals>.do_confirm_resize" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG nova.compute.manager [None req-1dd8654d-0753-439b-baa4-fe52888fd4cf tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Going to confirm migration 2 {{(pid=72081) do_confirm_resize /opt/stack/nova/nova/compute/manager.py:4663}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1dd8654d-0753-439b-baa4-fe52888fd4cf tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "refresh_cache-65ba7373-1427-409d-9aa7-6cb7894d665b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1dd8654d-0753-439b-baa4-fe52888fd4cf tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquired lock "refresh_cache-65ba7373-1427-409d-9aa7-6cb7894d665b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG nova.network.neutron [None req-1dd8654d-0753-439b-baa4-fe52888fd4cf tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG nova.objects.instance [None req-1dd8654d-0753-439b-baa4-fe52888fd4cf tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'info_cache' on Instance uuid 65ba7373-1427-409d-9aa7-6cb7894d665b {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.309s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] <== connect_volume: return (2311ms) {'type': 'block', 'scsi_wwn': '23564643731633831', 'path': '/dev/sdc'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23564643731633831', 'path': '/dev/sdc'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:53:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestShelveInstance-server-1000809041',display_name='tempest-TestShelveInstance-server-1000809041',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testshelveinstance-server-1000809041',id=42,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHHBsGFyn1IfBui/0BXjt7hpTkTH4PsZRTzQ4VM6GFP5pgRYv1Yo7P5WOmhK4SKKnd1P8T0z7p8gKrx3Yxp/sOlxdxioj+WGXT88SaOKal87PVel/g0WojZ8NRP1p7DX/g==',key_name='tempest-TestShelveInstance-1986276884',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='d8ec2318ea05403db924630461ea77e5',ramdisk_id='',reservation_id='r-3erg1hz6',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestShelveInstance-130328817',owner_user_name='tempest-TestShelveInstance-130328817-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:53:11Z,user_data=None,user_id='447437f9d71b4f8ab0da168f0fca732f',uuid=1e4dae15-f69f-4e44-959a-7480e185020b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "address": "fa:16:3e:e9:ff:d5", "network": {"id": "99c0b9a9-e1b4-43d1-bbbb-66560eca330a", "bridge": "br-int", "label": "tempest-TestShelveInstance-2092145606-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d8ec2318ea05403db924630461ea77e5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be56e0c-9e", "ovs_interfaceid": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Converting VIF {"id": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "address": "fa:16:3e:e9:ff:d5", "network": {"id": "99c0b9a9-e1b4-43d1-bbbb-66560eca330a", "bridge": "br-int", "label": "tempest-TestShelveInstance-2092145606-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d8ec2318ea05403db924630461ea77e5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be56e0c-9e", "ovs_interfaceid": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e9:ff:d5,bridge_name='br-int',has_traffic_filtering=True,id=1be56e0c-9ee5-4df6-bbd6-f866a4e54e90,network=Network(99c0b9a9-e1b4-43d1-bbbb-66560eca330a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1be56e0c-9e') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lazy-loading 'pci_devices' on Instance uuid 1e4dae15-f69f-4e44-959a-7480e185020b {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:53:17 user nova-compute[72081]:   <uuid>1e4dae15-f69f-4e44-959a-7480e185020b</uuid>
Apr 12 03:53:17 user nova-compute[72081]:   <name>instance-0000002a</name>
Apr 12 03:53:17 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:53:17 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:53:17 user nova-compute[72081]:   <metadata>
Apr 12 03:53:17 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:53:17 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:53:17 user nova-compute[72081]:       <nova:name>tempest-TestShelveInstance-server-1000809041</nova:name>
Apr 12 03:53:17 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:53:15</nova:creationTime>
Apr 12 03:53:17 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:53:17 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:53:17 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:53:17 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:53:17 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:53:17 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:53:17 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:53:17 user nova-compute[72081]:       <nova:owner>
Apr 12 03:53:17 user nova-compute[72081]:         <nova:user uuid="447437f9d71b4f8ab0da168f0fca732f">tempest-TestShelveInstance-130328817-project-member</nova:user>
Apr 12 03:53:17 user nova-compute[72081]:         <nova:project uuid="d8ec2318ea05403db924630461ea77e5">tempest-TestShelveInstance-130328817</nova:project>
Apr 12 03:53:17 user nova-compute[72081]:       </nova:owner>
Apr 12 03:53:17 user nova-compute[72081]:       <nova:ports>
Apr 12 03:53:17 user nova-compute[72081]:         <nova:port uuid="1be56e0c-9ee5-4df6-bbd6-f866a4e54e90">
Apr 12 03:53:17 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Apr 12 03:53:17 user nova-compute[72081]:         </nova:port>
Apr 12 03:53:17 user nova-compute[72081]:       </nova:ports>
Apr 12 03:53:17 user nova-compute[72081]:     </nova:instance>
Apr 12 03:53:17 user nova-compute[72081]:   </metadata>
Apr 12 03:53:17 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:53:17 user nova-compute[72081]:     <system>
Apr 12 03:53:17 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:53:17 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:53:17 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:53:17 user nova-compute[72081]:       <entry name="serial">1e4dae15-f69f-4e44-959a-7480e185020b</entry>
Apr 12 03:53:17 user nova-compute[72081]:       <entry name="uuid">1e4dae15-f69f-4e44-959a-7480e185020b</entry>
Apr 12 03:53:17 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:53:17 user nova-compute[72081]:     </system>
Apr 12 03:53:17 user nova-compute[72081]:   </sysinfo>
Apr 12 03:53:17 user nova-compute[72081]:   <os>
Apr 12 03:53:17 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:53:17 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:53:17 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:53:17 user nova-compute[72081]:   </os>
Apr 12 03:53:17 user nova-compute[72081]:   <features>
Apr 12 03:53:17 user nova-compute[72081]:     <acpi/>
Apr 12 03:53:17 user nova-compute[72081]:     <apic/>
Apr 12 03:53:17 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:53:17 user nova-compute[72081]:   </features>
Apr 12 03:53:17 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:53:17 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:53:17 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:53:17 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:53:17 user nova-compute[72081]:   </clock>
Apr 12 03:53:17 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:53:17 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:53:17 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:53:17 user nova-compute[72081]:   </cpu>
Apr 12 03:53:17 user nova-compute[72081]:   <devices>
Apr 12 03:53:17 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:53:17 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:53:17 user nova-compute[72081]:       <source dev="/dev/sdc"/>
Apr 12 03:53:17 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:53:17 user nova-compute[72081]:       <serial>d6a07e3a-be2a-4191-b1b7-09667064fb02</serial>
Apr 12 03:53:17 user nova-compute[72081]:     </disk>
Apr 12 03:53:17 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:53:17 user nova-compute[72081]:       <mac address="fa:16:3e:e9:ff:d5"/>
Apr 12 03:53:17 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:53:17 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:53:17 user nova-compute[72081]:       <target dev="tap1be56e0c-9e"/>
Apr 12 03:53:17 user nova-compute[72081]:     </interface>
Apr 12 03:53:17 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:53:17 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/1e4dae15-f69f-4e44-959a-7480e185020b/console.log" append="off"/>
Apr 12 03:53:17 user nova-compute[72081]:     </serial>
Apr 12 03:53:17 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:53:17 user nova-compute[72081]:     <video>
Apr 12 03:53:17 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:53:17 user nova-compute[72081]:     </video>
Apr 12 03:53:17 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:53:17 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:53:17 user nova-compute[72081]:     </rng>
Apr 12 03:53:17 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:53:17 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:53:17 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:53:17 user nova-compute[72081]:     </memballoon>
Apr 12 03:53:17 user nova-compute[72081]:   </devices>
Apr 12 03:53:17 user nova-compute[72081]: </domain>
Apr 12 03:53:17 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:53:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestShelveInstance-server-1000809041',display_name='tempest-TestShelveInstance-server-1000809041',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testshelveinstance-server-1000809041',id=42,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHHBsGFyn1IfBui/0BXjt7hpTkTH4PsZRTzQ4VM6GFP5pgRYv1Yo7P5WOmhK4SKKnd1P8T0z7p8gKrx3Yxp/sOlxdxioj+WGXT88SaOKal87PVel/g0WojZ8NRP1p7DX/g==',key_name='tempest-TestShelveInstance-1986276884',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='d8ec2318ea05403db924630461ea77e5',ramdisk_id='',reservation_id='r-3erg1hz6',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestShelveInstance-130328817',owner_user_name='tempest-TestShelveInstance-130328817-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:53:11Z,user_data=None,user_id='447437f9d71b4f8ab0da168f0fca732f',uuid=1e4dae15-f69f-4e44-959a-7480e185020b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "address": "fa:16:3e:e9:ff:d5", "network": {"id": "99c0b9a9-e1b4-43d1-bbbb-66560eca330a", "bridge": "br-int", "label": "tempest-TestShelveInstance-2092145606-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d8ec2318ea05403db924630461ea77e5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be56e0c-9e", "ovs_interfaceid": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Converting VIF {"id": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "address": "fa:16:3e:e9:ff:d5", "network": {"id": "99c0b9a9-e1b4-43d1-bbbb-66560eca330a", "bridge": "br-int", "label": "tempest-TestShelveInstance-2092145606-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d8ec2318ea05403db924630461ea77e5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be56e0c-9e", "ovs_interfaceid": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e9:ff:d5,bridge_name='br-int',has_traffic_filtering=True,id=1be56e0c-9ee5-4df6-bbd6-f866a4e54e90,network=Network(99c0b9a9-e1b4-43d1-bbbb-66560eca330a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1be56e0c-9e') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG os_vif [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:e9:ff:d5,bridge_name='br-int',has_traffic_filtering=True,id=1be56e0c-9ee5-4df6-bbd6-f866a4e54e90,network=Network(99c0b9a9-e1b4-43d1-bbbb-66560eca330a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1be56e0c-9e') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap1be56e0c-9e, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap1be56e0c-9e, col_values=(('external_ids', {'iface-id': '1be56e0c-9ee5-4df6-bbd6-f866a4e54e90', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:e9:ff:d5', 'vm-uuid': '1e4dae15-f69f-4e44-959a-7480e185020b'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:17 user nova-compute[72081]: INFO os_vif [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:e9:ff:d5,bridge_name='br-int',has_traffic_filtering=True,id=1be56e0c-9ee5-4df6-bbd6-f866a4e54e90,network=Network(99c0b9a9-e1b4-43d1-bbbb-66560eca330a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1be56e0c-9e')
Apr 12 03:53:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:53:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] No VIF found with MAC fa:16:3e:e9:ff:d5, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:53:18 user nova-compute[72081]: DEBUG nova.network.neutron [None req-1dd8654d-0753-439b-baa4-fe52888fd4cf tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Updating instance_info_cache with network_info: [{"id": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "address": "fa:16:3e:e9:20:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e82a6e-18", "ovs_interfaceid": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:53:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1dd8654d-0753-439b-baa4-fe52888fd4cf tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Releasing lock "refresh_cache-65ba7373-1427-409d-9aa7-6cb7894d665b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:53:18 user nova-compute[72081]: DEBUG nova.objects.instance [None req-1dd8654d-0753-439b-baa4-fe52888fd4cf tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'migration_context' on Instance uuid 65ba7373-1427-409d-9aa7-6cb7894d665b {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:53:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1dd8654d-0753-439b-baa4-fe52888fd4cf tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_source" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:53:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1dd8654d-0753-439b-baa4-fe52888fd4cf tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_source" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:53:18 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-1dd8654d-0753-439b-baa4-fe52888fd4cf tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:53:18 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-1dd8654d-0753-439b-baa4-fe52888fd4cf tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:53:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1dd8654d-0753-439b-baa4-fe52888fd4cf tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_source" :: held 0.286s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:53:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:18 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-1dd8654d-0753-439b-baa4-fe52888fd4cf tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Deleted allocation for migration 08dbfa70-c3f3-44ff-8dd8-3ea443fb7244
Apr 12 03:53:18 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:53:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1dd8654d-0753-439b-baa4-fe52888fd4cf tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b" "released" by "nova.compute.manager.ComputeManager.confirm_resize.<locals>.do_confirm_resize" :: held 1.457s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:53:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:19 user nova-compute[72081]: DEBUG nova.compute.manager [req-f47ac643-e953-44bd-8a91-6b5170b5e4f1 req-f6d79c9e-adb2-4c38-b7a9-0a65d5bf5659 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Received event network-vif-plugged-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:53:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f47ac643-e953-44bd-8a91-6b5170b5e4f1 req-f6d79c9e-adb2-4c38-b7a9-0a65d5bf5659 service nova] Acquiring lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:53:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f47ac643-e953-44bd-8a91-6b5170b5e4f1 req-f6d79c9e-adb2-4c38-b7a9-0a65d5bf5659 service nova] Lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:53:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f47ac643-e953-44bd-8a91-6b5170b5e4f1 req-f6d79c9e-adb2-4c38-b7a9-0a65d5bf5659 service nova] Lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:53:19 user nova-compute[72081]: DEBUG nova.compute.manager [req-f47ac643-e953-44bd-8a91-6b5170b5e4f1 req-f6d79c9e-adb2-4c38-b7a9-0a65d5bf5659 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] No waiting events found dispatching network-vif-plugged-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:53:19 user nova-compute[72081]: WARNING nova.compute.manager [req-f47ac643-e953-44bd-8a91-6b5170b5e4f1 req-f6d79c9e-adb2-4c38-b7a9-0a65d5bf5659 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Received unexpected event network-vif-plugged-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 for instance with vm_state building and task_state spawning.
Apr 12 03:53:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:21 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271601.313394, 1e4dae15-f69f-4e44-959a-7480e185020b => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:53:21 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] VM Resumed (Lifecycle Event)
Apr 12 03:53:21 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:53:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:53:21 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Instance spawned successfully.
Apr 12 03:53:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:53:21 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:53:21 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:53:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:53:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:53:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:53:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:53:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:53:21 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:53:21 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:53:21 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271601.314818, 1e4dae15-f69f-4e44-959a-7480e185020b => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:53:21 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] VM Started (Lifecycle Event)
Apr 12 03:53:21 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:53:21 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:53:21 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:53:21 user nova-compute[72081]: INFO nova.compute.manager [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Took 6.03 seconds to spawn the instance on the hypervisor.
Apr 12 03:53:21 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:53:21 user nova-compute[72081]: INFO nova.compute.manager [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Took 11.25 seconds to build instance.
Apr 12 03:53:21 user nova-compute[72081]: DEBUG nova.compute.manager [req-2b74ed58-b883-4a81-b265-3c622812b0df req-3f1e2d9b-ec46-4c72-8e71-dc6facb6fb7f service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Received event network-vif-plugged-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:53:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2b74ed58-b883-4a81-b265-3c622812b0df req-3f1e2d9b-ec46-4c72-8e71-dc6facb6fb7f service nova] Acquiring lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:53:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2b74ed58-b883-4a81-b265-3c622812b0df req-3f1e2d9b-ec46-4c72-8e71-dc6facb6fb7f service nova] Lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:53:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2b74ed58-b883-4a81-b265-3c622812b0df req-3f1e2d9b-ec46-4c72-8e71-dc6facb6fb7f service nova] Lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:53:21 user nova-compute[72081]: DEBUG nova.compute.manager [req-2b74ed58-b883-4a81-b265-3c622812b0df req-3f1e2d9b-ec46-4c72-8e71-dc6facb6fb7f service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] No waiting events found dispatching network-vif-plugged-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:53:21 user nova-compute[72081]: WARNING nova.compute.manager [req-2b74ed58-b883-4a81-b265-3c622812b0df req-3f1e2d9b-ec46-4c72-8e71-dc6facb6fb7f service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Received unexpected event network-vif-plugged-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 for instance with vm_state active and task_state None.
Apr 12 03:53:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4d2f6ea0-c61a-4db0-bde4-915c975ea9cb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "1e4dae15-f69f-4e44-959a-7480e185020b" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 11.371s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:53:21 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:53:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:53:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:53:21 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:53:21 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:53:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9a705900-a0de-46e3-9af4-d07911f181c0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:53:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9a705900-a0de-46e3-9af4-d07911f181c0/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:53:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9a705900-a0de-46e3-9af4-d07911f181c0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:53:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9a705900-a0de-46e3-9af4-d07911f181c0/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:53:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:53:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:53:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:53:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:53:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:53:22 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:53:22 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:53:22 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:53:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8487MB free_disk=26.03534698486328GB free_vcpus=8 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:53:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:53:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:53:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:53:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 65ba7373-1427-409d-9aa7-6cb7894d665b actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 192, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:53:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 9a705900-a0de-46e3-9af4-d07911f181c0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:53:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 1e4dae15-f69f-4e44-959a-7480e185020b actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:53:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 4 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:53:22 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=1088MB phys_disk=40GB used_disk=2GB total_vcpus=12 used_vcpus=4 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:53:23 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:53:23 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:53:23 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:53:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.304s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:53:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:24 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:53:24 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:53:24 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:53:24 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:53:25 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:53:26 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:53:27 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:53:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:53:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-65ba7373-1427-409d-9aa7-6cb7894d665b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:53:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-65ba7373-1427-409d-9aa7-6cb7894d665b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:53:27 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 03:53:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:28 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Updating instance_info_cache with network_info: [{"id": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "address": "fa:16:3e:e9:20:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e82a6e-18", "ovs_interfaceid": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:53:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-65ba7373-1427-409d-9aa7-6cb7894d665b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:53:28 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 03:53:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:53:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:53:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 03:53:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 03:53:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 03:53:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:53:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:54:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:54:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:54:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 03:54:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 03:54:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 03:54:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:54:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:54:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:54:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:54:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:54:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 03:54:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 03:54:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 03:54:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:54:14 user nova-compute[72081]: DEBUG nova.compute.manager [req-4ab01cda-3b5e-4701-bb91-d818e08d9363 req-527ed214-5302-41de-8962-8aecfbd070f4 service nova] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Received event network-changed-55b4c29c-432e-4323-a5d6-fc62a42f8655 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:54:14 user nova-compute[72081]: DEBUG nova.compute.manager [req-4ab01cda-3b5e-4701-bb91-d818e08d9363 req-527ed214-5302-41de-8962-8aecfbd070f4 service nova] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Refreshing instance network info cache due to event network-changed-55b4c29c-432e-4323-a5d6-fc62a42f8655. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:54:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4ab01cda-3b5e-4701-bb91-d818e08d9363 req-527ed214-5302-41de-8962-8aecfbd070f4 service nova] Acquiring lock "refresh_cache-9a705900-a0de-46e3-9af4-d07911f181c0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:54:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4ab01cda-3b5e-4701-bb91-d818e08d9363 req-527ed214-5302-41de-8962-8aecfbd070f4 service nova] Acquired lock "refresh_cache-9a705900-a0de-46e3-9af4-d07911f181c0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:54:14 user nova-compute[72081]: DEBUG nova.network.neutron [req-4ab01cda-3b5e-4701-bb91-d818e08d9363 req-527ed214-5302-41de-8962-8aecfbd070f4 service nova] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Refreshing network info cache for port 55b4c29c-432e-4323-a5d6-fc62a42f8655 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:54:15 user nova-compute[72081]: DEBUG nova.network.neutron [req-4ab01cda-3b5e-4701-bb91-d818e08d9363 req-527ed214-5302-41de-8962-8aecfbd070f4 service nova] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Updated VIF entry in instance network info cache for port 55b4c29c-432e-4323-a5d6-fc62a42f8655. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:54:15 user nova-compute[72081]: DEBUG nova.network.neutron [req-4ab01cda-3b5e-4701-bb91-d818e08d9363 req-527ed214-5302-41de-8962-8aecfbd070f4 service nova] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Updating instance_info_cache with network_info: [{"id": "55b4c29c-432e-4323-a5d6-fc62a42f8655", "address": "fa:16:3e:e8:16:de", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.101", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55b4c29c-43", "ovs_interfaceid": "55b4c29c-432e-4323-a5d6-fc62a42f8655", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:54:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4ab01cda-3b5e-4701-bb91-d818e08d9363 req-527ed214-5302-41de-8962-8aecfbd070f4 service nova] Releasing lock "refresh_cache-9a705900-a0de-46e3-9af4-d07911f181c0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "9a705900-a0de-46e3-9af4-d07911f181c0" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "9a705900-a0de-46e3-9af4-d07911f181c0" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG nova.objects.instance [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'flavor' on Instance uuid 9a705900-a0de-46e3-9af4-d07911f181c0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "9a705900-a0de-46e3-9af4-d07911f181c0" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.052s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "9a705900-a0de-46e3-9af4-d07911f181c0" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "9a705900-a0de-46e3-9af4-d07911f181c0" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:54:16 user nova-compute[72081]: INFO nova.compute.manager [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Attaching volume 952561b4-1ccc-4f15-b53b-8fcf4189931e to /dev/vdb
Apr 12 03:54:16 user nova-compute[72081]: DEBUG os_brick.utils [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f69bc93f-301a-4c3b-83ea-23d0546bb7e9]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[79cf511c-89b5-40f8-a71f-ce534f3c0d1e]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6183d23b-b7df-45b1-9dae-c4485714e06f]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2cdb7de4-19f2-4416-8c9c-be49ba3deb36]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:54:16 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:54:16 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e602ee57-1ef0-4aea-922b-4c3f9840c5ca]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG os_brick.utils [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] <== get_connector_properties: return (135ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:54:16 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Updating existing volume attachment record: 5994ee94-096f-4894-9d63-60d05df1399c {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:54:17 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:54:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:54:18 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:54:19 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'vJQMKmws', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:54:20 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:54:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:54:20 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:54:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:54:20 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:54:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bd66a3c7-3db4-4466-896e-2276b34ffde5]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:54:20 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7124574d-663f-403c-87d0-a131b86ee1fa]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e] added
Apr 12 03:54:20 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1b867299-7087-4afc-a22b-ba909fd2d0a0]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb23261f-6072-4ed5-bbae-bab547e33acc]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v vJQMKmws {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v vJQMKmws" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[647ccb7f-477f-42a5-86d0-d3c09394cb35]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'vJQMKmws'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[201d3752-0f8d-4772-a3d3-381fad25a5cd]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6dd7527-8a4e-4da1-a61f-163465976bc7]: (4, ('tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:54:20 user nova-compute[72081]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)
Apr 12 03:54:20 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsi session list stdout=tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:54:20 user nova-compute[72081]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)
Apr 12 03:54:20 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e -p 172.16.0.220:3260 --login" returned: 0 in 0.054s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7a7b4271-db99-455e-a17d-4a237618ff6b]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e, portal: 172.16.0.220,3260]
Apr 12 03:54:20 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e, portal: 172.16.0.220,3260] successful.
Apr 12 03:54:20 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:54:20 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:21 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c8d978da-39f7-464c-8de9-40bfc47a90c5]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:21 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:54:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:21 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[38f4a58a-9f00-4473-b595-5cc59e4515c8]: (4, ('tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:21 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:54:21 user nova-compute[72081]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)
Apr 12 03:54:21 user nova-compute[72081]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e (non-flash)
Apr 12 03:54:21 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:54:21 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsi session list stdout=tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:54:21 user nova-compute[72081]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)
Apr 12 03:54:21 user nova-compute[72081]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e (non-flash)
Apr 12 03:54:21 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:54:21 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.265s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:54:21 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:54:21 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] HCTL ('35', '-', '-', 0) found on session 47 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:54:21 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:54:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:21 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:21 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4c65b61a-999d-4683-a662-1b70e2d6bed2]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:21 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Searching for a device in session 47 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:54:21 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'vJQMKmws', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:54:22 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.297s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:54:22 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] <== connect_volume: return (1299ms) {'type': 'block', 'scsi_wwn': '23563336132313335', 'path': '/dev/sdd'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:54:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23563336132313335', 'path': '/dev/sdd'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:54:22 user nova-compute[72081]: DEBUG nova.objects.instance [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'flavor' on Instance uuid 9a705900-a0de-46e3-9af4-d07911f181c0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:54:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] attach device xml: <disk type="block" device="disk">
Apr 12 03:54:22 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:54:22 user nova-compute[72081]:   <source dev="/dev/sdd"/>
Apr 12 03:54:22 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:54:22 user nova-compute[72081]:   <serial>952561b4-1ccc-4f15-b53b-8fcf4189931e</serial>
Apr 12 03:54:22 user nova-compute[72081]: </disk>
Apr 12 03:54:22 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 03:54:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:54:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:54:22 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] No VIF found with MAC fa:16:3e:e8:16:de, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:54:22 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-671c8004-56bc-4125-bf4e-a768d582c946 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "9a705900-a0de-46e3-9af4-d07911f181c0" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.125s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:54:22 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:54:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:54:23 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:54:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:54:23 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:54:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:54:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:54:23 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:54:23 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:54:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9a705900-a0de-46e3-9af4-d07911f181c0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9a705900-a0de-46e3-9af4-d07911f181c0/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:23 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9a705900-a0de-46e3-9af4-d07911f181c0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9a705900-a0de-46e3-9af4-d07911f181c0/disk --force-share --output=json" returned: 0 in 0.168s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:24 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdd (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:54:24 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:54:24 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:54:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json" returned: 0 in 0.177s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json" returned: 0 in 0.156s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "9a705900-a0de-46e3-9af4-d07911f181c0" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:54:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "9a705900-a0de-46e3-9af4-d07911f181c0" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.003s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:54:25 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:54:25 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:54:25 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8488MB free_disk=26.034194946289062GB free_vcpus=8 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG nova.objects.instance [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'flavor' on Instance uuid 9a705900-a0de-46e3-9af4-d07911f181c0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "9a705900-a0de-46e3-9af4-d07911f181c0" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.108s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 65ba7373-1427-409d-9aa7-6cb7894d665b actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 192, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 9a705900-a0de-46e3-9af4-d07911f181c0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 1e4dae15-f69f-4e44-959a-7480e185020b actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 4 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=1088MB phys_disk=40GB used_disk=2GB total_vcpus=12 used_vcpus=4 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.286s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "9a705900-a0de-46e3-9af4-d07911f181c0" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "9a705900-a0de-46e3-9af4-d07911f181c0" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:54:25 user nova-compute[72081]: INFO nova.compute.manager [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Attaching volume 3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 to /dev/vdc
Apr 12 03:54:25 user nova-compute[72081]: DEBUG os_brick.utils [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3eb61379-c216-4f33-a0d6-7278814c1966]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[54846de5-27f7-45f5-a292-143bd33cc5b8]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6b7a8c1e-8bc1-4d7d-b015-abfd3ff8c44e]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b8241413-d238-4c59-a5f4-176317275be5]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:54:25 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:54:25 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[543822c7-ce48-4e7a-979b-52b0a713e3c4]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG os_brick.utils [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] <== get_connector_properties: return (124ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:54:25 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Updating existing volume attachment record: 0f0f1e97-1b6a-480b-b1d1-0751d4737d87 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:54:26 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:54:26 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:54:26 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:54:27 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:54:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:54:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Rebuilding the list of instances to heal {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 12 03:54:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-8d533f5f-dbed-4878-a9b1-2ce4fae8fa13" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:54:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-8d533f5f-dbed-4878-a9b1-2ce4fae8fa13" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:54:27 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 03:54:27 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lazy-loading 'info_cache' on Instance uuid 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:54:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:54:28 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Updating instance_info_cache with network_info: [{"id": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "address": "fa:16:3e:e2:fd:44", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2ef93926-6c", "ovs_interfaceid": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:54:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-8d533f5f-dbed-4878-a9b1-2ce4fae8fa13" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:54:28 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 03:54:28 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:54:28 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'SzfAIvel', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:54:29 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:54:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:54:29 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:54:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:54:29 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6195db1-dc8b-41cf-b36e-278ed64159b9]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:54:29 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7c18c7ac-ebad-4a1a-80a7-d7a32208b4c2]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611] added
Apr 12 03:54:29 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2ba52818-9aa7-419a-b55d-e1950aeec5cf]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[59519d32-37b7-4a4c-9c30-c9d87089e41b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v SzfAIvel {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v SzfAIvel" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5469ec31-b521-46db-b961-3d3d8fbe5a7b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'SzfAIvel'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[dcee20a2-4be4-49e7-9080-d0cf851c604e]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[eed0e775-e8b1-4cde-bf35-9088272fff22]: (4, ('tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:54:29 user nova-compute[72081]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)
Apr 12 03:54:29 user nova-compute[72081]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e (non-flash)
Apr 12 03:54:29 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsi session list stdout=tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:54:29 user nova-compute[72081]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)
Apr 12 03:54:29 user nova-compute[72081]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e (non-flash)
Apr 12 03:54:29 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 -p 172.16.0.220:3260 --login" returned: 0 in 0.058s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b4469978-62f8-4b11-bf24-56fe451a76b6]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611, portal: 172.16.0.220,3260]
Apr 12 03:54:29 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611, portal: 172.16.0.220,3260] successful.
Apr 12 03:54:29 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[353b80ac-7731-41b7-aff6-b749397fa3aa]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4788e95e-69b3-4839-a6fa-0f0b75652955]: (4, ('tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:54:29 user nova-compute[72081]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)
Apr 12 03:54:29 user nova-compute[72081]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e (non-flash)
Apr 12 03:54:29 user nova-compute[72081]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 (non-flash)
Apr 12 03:54:29 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsi session list stdout=tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:54:29 user nova-compute[72081]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)
Apr 12 03:54:29 user nova-compute[72081]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e (non-flash)
Apr 12 03:54:29 user nova-compute[72081]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 (non-flash)
Apr 12 03:54:29 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.280s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] HCTL ('36', '-', '-', 0) found on session 48 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b0b7a438-3502-47b4-8c63-f335f17b0ff7]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Searching for a device in session 48 and hctl ['36', '*', '*', 0] yield: sde {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:54:29 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'SzfAIvel', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:54:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.315s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:54:30 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] <== connect_volume: return (1317ms) {'type': 'block', 'scsi_wwn': '23334626539646139', 'path': '/dev/sde'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:54:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23334626539646139', 'path': '/dev/sde'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:54:30 user nova-compute[72081]: DEBUG nova.objects.instance [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'flavor' on Instance uuid 9a705900-a0de-46e3-9af4-d07911f181c0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:54:30 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] attach device xml: <disk type="block" device="disk">
Apr 12 03:54:30 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:54:30 user nova-compute[72081]:   <source dev="/dev/sde"/>
Apr 12 03:54:30 user nova-compute[72081]:   <target dev="vdc" bus="virtio"/>
Apr 12 03:54:30 user nova-compute[72081]:   <serial>3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611</serial>
Apr 12 03:54:30 user nova-compute[72081]: </disk>
Apr 12 03:54:30 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 03:54:31 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:54:31 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:54:31 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] No BDM found with device name vdc, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:54:31 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] No VIF found with MAC fa:16:3e:e8:16:de, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:54:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-722af097-bccf-4fdc-84ce-9da6c018c117 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "9a705900-a0de-46e3-9af4-d07911f181c0" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.186s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "9a705900-a0de-46e3-9af4-d07911f181c0" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "9a705900-a0de-46e3-9af4-d07911f181c0" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:54:32 user nova-compute[72081]: INFO nova.compute.manager [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Detaching volume 952561b4-1ccc-4f15-b53b-8fcf4189931e
Apr 12 03:54:32 user nova-compute[72081]: INFO nova.virt.block_device [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Attempting to driver detach volume 952561b4-1ccc-4f15-b53b-8fcf4189931e from mountpoint /dev/vdb
Apr 12 03:54:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Attempting to detach device vdb from instance 9a705900-a0de-46e3-9af4-d07911f181c0 from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:54:32 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:54:32 user nova-compute[72081]:   <source dev="/dev/sdd"/>
Apr 12 03:54:32 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:54:32 user nova-compute[72081]:   <serial>952561b4-1ccc-4f15-b53b-8fcf4189931e</serial>
Apr 12 03:54:32 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 03:54:32 user nova-compute[72081]: </disk>
Apr 12 03:54:32 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:54:32 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Successfully detached device vdb from instance 9a705900-a0de-46e3-9af4-d07911f181c0 from the persistent domain config.
Apr 12 03:54:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 9a705900-a0de-46e3-9af4-d07911f181c0 from the live domain config. {{(pid=72081) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:54:32 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:54:32 user nova-compute[72081]:   <source dev="/dev/sdd"/>
Apr 12 03:54:32 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 03:54:32 user nova-compute[72081]:   <serial>952561b4-1ccc-4f15-b53b-8fcf4189931e</serial>
Apr 12 03:54:32 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 03:54:32 user nova-compute[72081]: </disk>
Apr 12 03:54:32 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Received event <DeviceRemovedEvent: 1681271672.565574, 9a705900-a0de-46e3-9af4-d07911f181c0 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72081) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 9a705900-a0de-46e3-9af4-d07911f181c0 {{(pid=72081) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 12 03:54:32 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Successfully detached device vdb from instance 9a705900-a0de-46e3-9af4-d07911f181c0 from the live domain config.
Apr 12 03:54:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'vJQMKmws', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:54:32 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:54:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[470ba0e3-d679-4a44-8acc-ac98be97fec5]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:54:32 user nova-compute[72081]: No targets found.
Apr 12 03:54:32 user nova-compute[72081]: iSNS:
Apr 12 03:54:32 user nova-compute[72081]: No targets found.
Apr 12 03:54:32 user nova-compute[72081]: STATIC:
Apr 12 03:54:32 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd
Apr 12 03:54:32 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:54:32 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:54:32 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611
Apr 12 03:54:32 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:54:32 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:54:32 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e
Apr 12 03:54:32 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:54:32 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:54:32 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02
Apr 12 03:54:32 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:54:32 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:54:32 user nova-compute[72081]: FIRMWARE:
Apr 12 03:54:32 user nova-compute[72081]: No targets found.
Apr 12 03:54:32 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:54:32 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:54:32 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[954df5c3-fc9b-48b0-8268-ca6c75b975b5]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2f252128-6283-4529-9c17-05cd235508a9]: (4, ('tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:54:32 user nova-compute[72081]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)
Apr 12 03:54:32 user nova-compute[72081]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e (non-flash)
Apr 12 03:54:32 user nova-compute[72081]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 (non-flash)
Apr 12 03:54:32 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsi session list stdout=tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:54:32 user nova-compute[72081]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)
Apr 12 03:54:32 user nova-compute[72081]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e (non-flash)
Apr 12 03:54:32 user nova-compute[72081]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 (non-flash)
Apr 12 03:54:32 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fb6fa84c0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e'): ({'sdd'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Removing single pathed devices sdd {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[cccaa1a4-4d73-48d5-a498-c3983310c752]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[da290fba-1cc6-428f-bdb2-810ee95e8839]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Flushing IO for device /dev/sdd {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.013s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d7e0f773-89aa-43b8-b632-7b2838e8158a]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.041s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4cf44e17-de7c-48a1-90e5-7a3eb204a53f]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] SCSI volumes sdd have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4a07c77b-bcd6-4db4-a6f6-be4e1a4e88ee]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e -p 172.16.0.220:3260 --logout" returned: 0 in 0.064s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[520393c0-826f-4143-bb1e-44f42995e345]: (4, ('Logging out of session [sid: 47, target: iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e, portal: 172.16.0.220,3260]\nLogout of [sid: 47, target: iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 47, target: iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e, portal: 172.16.0.220,3260]
Apr 12 03:54:32 user nova-compute[72081]: Logout of [sid: 47, target: iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e, portal: 172.16.0.220,3260] successful.
Apr 12 03:54:32 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:952561b4-1ccc-4f15-b53b-8fcf4189931e -p 172.16.0.220:3260 --op delete" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[011fbbd9-c6a7-4252-9189-0f6e599b38b0]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.303s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] <== disconnect_volume: return (304ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 03:54:32 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:54:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:54:33 user nova-compute[72081]: DEBUG nova.objects.instance [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'flavor' on Instance uuid 9a705900-a0de-46e3-9af4-d07911f181c0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:54:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a05beafe-fbc7-4087-be77-c83439e7eb3e tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "9a705900-a0de-46e3-9af4-d07911f181c0" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.628s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:54:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "9a705900-a0de-46e3-9af4-d07911f181c0" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:54:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "9a705900-a0de-46e3-9af4-d07911f181c0" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:54:34 user nova-compute[72081]: INFO nova.compute.manager [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Detaching volume 3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611
Apr 12 03:54:34 user nova-compute[72081]: INFO nova.virt.block_device [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Attempting to driver detach volume 3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 from mountpoint /dev/vdc
Apr 12 03:54:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Attempting to detach device vdc from instance 9a705900-a0de-46e3-9af4-d07911f181c0 from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 03:54:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:54:34 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:54:34 user nova-compute[72081]:   <source dev="/dev/sde"/>
Apr 12 03:54:34 user nova-compute[72081]:   <target dev="vdc" bus="virtio"/>
Apr 12 03:54:34 user nova-compute[72081]:   <serial>3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611</serial>
Apr 12 03:54:34 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
Apr 12 03:54:34 user nova-compute[72081]: </disk>
Apr 12 03:54:34 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:54:34 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Successfully detached device vdc from instance 9a705900-a0de-46e3-9af4-d07911f181c0 from the persistent domain config.
Apr 12 03:54:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] (1/8): Attempting to detach device vdc with device alias virtio-disk2 from instance 9a705900-a0de-46e3-9af4-d07911f181c0 from the live domain config. {{(pid=72081) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 12 03:54:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 03:54:34 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:54:34 user nova-compute[72081]:   <source dev="/dev/sde"/>
Apr 12 03:54:34 user nova-compute[72081]:   <target dev="vdc" bus="virtio"/>
Apr 12 03:54:34 user nova-compute[72081]:   <serial>3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611</serial>
Apr 12 03:54:34 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
Apr 12 03:54:34 user nova-compute[72081]: </disk>
Apr 12 03:54:34 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 03:54:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Received event <DeviceRemovedEvent: 1681271674.9044495, 9a705900-a0de-46e3-9af4-d07911f181c0 => virtio-disk2> from libvirt while the driver is waiting for it; dispatched. {{(pid=72081) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 12 03:54:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Start waiting for the detach event from libvirt for device vdc with device alias virtio-disk2 for instance 9a705900-a0de-46e3-9af4-d07911f181c0 {{(pid=72081) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 12 03:54:34 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Successfully detached device vdc from instance 9a705900-a0de-46e3-9af4-d07911f181c0 from the live domain config.
Apr 12 03:54:34 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:54:34 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'SzfAIvel', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:54:34 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:54:34 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:54:34 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:54:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:34 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0ceee6f6-a424-4625-a612-602efb9e0e2e]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:34 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:54:34 user nova-compute[72081]: No targets found.
Apr 12 03:54:34 user nova-compute[72081]: iSNS:
Apr 12 03:54:34 user nova-compute[72081]: No targets found.
Apr 12 03:54:34 user nova-compute[72081]: STATIC:
Apr 12 03:54:34 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd
Apr 12 03:54:34 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:54:34 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:54:34 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611
Apr 12 03:54:34 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:54:34 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:54:34 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02
Apr 12 03:54:34 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:54:34 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:54:34 user nova-compute[72081]: FIRMWARE:
Apr 12 03:54:34 user nova-compute[72081]: No targets found.
Apr 12 03:54:34 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:54:34 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:54:34 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:54:34 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:54:34 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:54:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:34 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[24dfd707-d593-4ca4-87d2-131bcb2d60cd]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:34 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[78ba0901-6d81-4969-8a66-3409efb2402b]: (4, ('tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:34 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:54:34 user nova-compute[72081]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)
Apr 12 03:54:34 user nova-compute[72081]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 (non-flash)
Apr 12 03:54:34 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:54:34 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsi session list stdout=tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:54:34 user nova-compute[72081]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)
Apr 12 03:54:34 user nova-compute[72081]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 (non-flash)
Apr 12 03:54:34 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:54:34 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3f747349d0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611'): ({'sde'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:54:34 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Removing single pathed devices sde {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:54:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[75496f0d-4dee-441a-b210-a4e1acfe21aa]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c0dcdf73-d657-4492-8828-e545e5d39116]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Flushing IO for device /dev/sde {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.013s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[37cf86e1-bc2f-4916-8d1d-792ed0c545d6]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.042s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7a9f0cbe-ccac-4612-9666-0239d16ae54e]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] SCSI volumes sde have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[53cd390e-cde2-430b-8213-85f1cae79f3e]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 -p 172.16.0.220:3260 --logout" returned: 0 in 0.061s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f5346e95-1a63-4be5-bc02-94df6d9950d7]: (4, ('Logging out of session [sid: 48, target: iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611, portal: 172.16.0.220,3260]\nLogout of [sid: 48, target: iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 48, target: iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611, portal: 172.16.0.220,3260]
Apr 12 03:54:35 user nova-compute[72081]: Logout of [sid: 48, target: iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611, portal: 172.16.0.220,3260] successful.
Apr 12 03:54:35 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3d2ece15-a8eb-4b1f-b0f5-cf2311fb6611 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[76d1acf2-0a37-42a5-8baf-e59a733e6ce0]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.296s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] <== disconnect_volume: return (297ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG nova.objects.instance [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'flavor' on Instance uuid 9a705900-a0de-46e3-9af4-d07911f181c0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:54:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-cce5a022-1201-4b66-8183-cc060b307a71 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "9a705900-a0de-46e3-9af4-d07911f181c0" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.558s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:54:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-da9eecbf-b46e-43ca-b681-1e690ada7113 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "9a705900-a0de-46e3-9af4-d07911f181c0" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:54:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-da9eecbf-b46e-43ca-b681-1e690ada7113 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "9a705900-a0de-46e3-9af4-d07911f181c0" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:54:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-da9eecbf-b46e-43ca-b681-1e690ada7113 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "9a705900-a0de-46e3-9af4-d07911f181c0-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:54:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-da9eecbf-b46e-43ca-b681-1e690ada7113 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "9a705900-a0de-46e3-9af4-d07911f181c0-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:54:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-da9eecbf-b46e-43ca-b681-1e690ada7113 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "9a705900-a0de-46e3-9af4-d07911f181c0-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:54:37 user nova-compute[72081]: INFO nova.compute.manager [None req-da9eecbf-b46e-43ca-b681-1e690ada7113 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Terminating instance
Apr 12 03:54:37 user nova-compute[72081]: DEBUG nova.compute.manager [None req-da9eecbf-b46e-43ca-b681-1e690ada7113 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:54:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:54:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:54:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:54:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-3008158b-8799-44d0-88f4-e921b3b56871 req-8f3ee86e-de06-48de-82e3-dd21d2031692 service nova] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Received event network-vif-unplugged-55b4c29c-432e-4323-a5d6-fc62a42f8655 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:54:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3008158b-8799-44d0-88f4-e921b3b56871 req-8f3ee86e-de06-48de-82e3-dd21d2031692 service nova] Acquiring lock "9a705900-a0de-46e3-9af4-d07911f181c0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:54:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3008158b-8799-44d0-88f4-e921b3b56871 req-8f3ee86e-de06-48de-82e3-dd21d2031692 service nova] Lock "9a705900-a0de-46e3-9af4-d07911f181c0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:54:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-3008158b-8799-44d0-88f4-e921b3b56871 req-8f3ee86e-de06-48de-82e3-dd21d2031692 service nova] Lock "9a705900-a0de-46e3-9af4-d07911f181c0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:54:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-3008158b-8799-44d0-88f4-e921b3b56871 req-8f3ee86e-de06-48de-82e3-dd21d2031692 service nova] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] No waiting events found dispatching network-vif-unplugged-55b4c29c-432e-4323-a5d6-fc62a42f8655 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:54:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-3008158b-8799-44d0-88f4-e921b3b56871 req-8f3ee86e-de06-48de-82e3-dd21d2031692 service nova] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Received event network-vif-unplugged-55b4c29c-432e-4323-a5d6-fc62a42f8655 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:54:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:54:37 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Instance destroyed successfully.
Apr 12 03:54:37 user nova-compute[72081]: DEBUG nova.objects.instance [None req-da9eecbf-b46e-43ca-b681-1e690ada7113 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lazy-loading 'resources' on Instance uuid 9a705900-a0de-46e3-9af4-d07911f181c0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:54:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-da9eecbf-b46e-43ca-b681-1e690ada7113 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:52:22Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1419019551',display_name='tempest-AttachVolumeTestJSON-server-1419019551',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1419019551',id=41,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBF5dfRZfMAh+PjjlIpXMM5gTNQ/AVHmZ6QANnWldIoSsGeWR4SRx5i4ef7EgAPnNzMv6qe56Q5J7su4WXqBoiAy/S0b9fCrjSIUZQ+BsTnQye+GfEYaIPsBJDEmU+yv9Hw==',key_name='tempest-keypair-1939295877',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:52:29Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='a341e582192a47548df8627d82c96ea8',ramdisk_id='',reservation_id='r-upbjpoau',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-617614045',owner_user_name='tempest-AttachVolumeTestJSON-617614045-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:52:30Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='872fbc3d59274cd480c17b4b7342499a',uuid=9a705900-a0de-46e3-9af4-d07911f181c0,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "55b4c29c-432e-4323-a5d6-fc62a42f8655", "address": "fa:16:3e:e8:16:de", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.101", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55b4c29c-43", "ovs_interfaceid": "55b4c29c-432e-4323-a5d6-fc62a42f8655", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:54:37 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-da9eecbf-b46e-43ca-b681-1e690ada7113 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Converting VIF {"id": "55b4c29c-432e-4323-a5d6-fc62a42f8655", "address": "fa:16:3e:e8:16:de", "network": {"id": "4bee7b8b-6409-4a1e-ab1e-480450f73fe5", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-1767481497-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.101", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a341e582192a47548df8627d82c96ea8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55b4c29c-43", "ovs_interfaceid": "55b4c29c-432e-4323-a5d6-fc62a42f8655", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:54:37 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-da9eecbf-b46e-43ca-b681-1e690ada7113 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:e8:16:de,bridge_name='br-int',has_traffic_filtering=True,id=55b4c29c-432e-4323-a5d6-fc62a42f8655,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap55b4c29c-43') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:54:37 user nova-compute[72081]: DEBUG os_vif [None req-da9eecbf-b46e-43ca-b681-1e690ada7113 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:e8:16:de,bridge_name='br-int',has_traffic_filtering=True,id=55b4c29c-432e-4323-a5d6-fc62a42f8655,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap55b4c29c-43') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:54:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:54:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap55b4c29c-43, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:54:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:54:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:54:37 user nova-compute[72081]: INFO os_vif [None req-da9eecbf-b46e-43ca-b681-1e690ada7113 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:e8:16:de,bridge_name='br-int',has_traffic_filtering=True,id=55b4c29c-432e-4323-a5d6-fc62a42f8655,network=Network(4bee7b8b-6409-4a1e-ab1e-480450f73fe5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap55b4c29c-43')
Apr 12 03:54:37 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-da9eecbf-b46e-43ca-b681-1e690ada7113 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Deleting instance files /opt/stack/data/nova/instances/9a705900-a0de-46e3-9af4-d07911f181c0_del
Apr 12 03:54:37 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-da9eecbf-b46e-43ca-b681-1e690ada7113 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Deletion of /opt/stack/data/nova/instances/9a705900-a0de-46e3-9af4-d07911f181c0_del complete
Apr 12 03:54:37 user nova-compute[72081]: INFO nova.compute.manager [None req-da9eecbf-b46e-43ca-b681-1e690ada7113 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Took 0.83 seconds to destroy the instance on the hypervisor.
Apr 12 03:54:37 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-da9eecbf-b46e-43ca-b681-1e690ada7113 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:54:37 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:54:37 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:54:38 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:54:38 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Took 0.87 seconds to deallocate network for instance.
Apr 12 03:54:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-9dce0e55-0867-4052-a2d1-c5145640aeef req-c0bdf65c-8b44-4d1f-b990-6b390ccd4d9e service nova] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Received event network-vif-deleted-55b4c29c-432e-4323-a5d6-fc62a42f8655 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:54:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-da9eecbf-b46e-43ca-b681-1e690ada7113 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:54:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-da9eecbf-b46e-43ca-b681-1e690ada7113 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:54:38 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-da9eecbf-b46e-43ca-b681-1e690ada7113 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:54:38 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-da9eecbf-b46e-43ca-b681-1e690ada7113 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:54:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-da9eecbf-b46e-43ca-b681-1e690ada7113 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.171s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:54:39 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-da9eecbf-b46e-43ca-b681-1e690ada7113 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Deleted allocations for instance 9a705900-a0de-46e3-9af4-d07911f181c0
Apr 12 03:54:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-da9eecbf-b46e-43ca-b681-1e690ada7113 tempest-AttachVolumeTestJSON-617614045 tempest-AttachVolumeTestJSON-617614045-project-member] Lock "9a705900-a0de-46e3-9af4-d07911f181c0" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.058s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:54:39 user nova-compute[72081]: DEBUG nova.compute.manager [req-10167fe7-c96d-428b-adb0-934029507ff6 req-037f1183-de42-4930-9ea4-eb92ee09844a service nova] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Received event network-vif-plugged-55b4c29c-432e-4323-a5d6-fc62a42f8655 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:54:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-10167fe7-c96d-428b-adb0-934029507ff6 req-037f1183-de42-4930-9ea4-eb92ee09844a service nova] Acquiring lock "9a705900-a0de-46e3-9af4-d07911f181c0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:54:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-10167fe7-c96d-428b-adb0-934029507ff6 req-037f1183-de42-4930-9ea4-eb92ee09844a service nova] Lock "9a705900-a0de-46e3-9af4-d07911f181c0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:54:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-10167fe7-c96d-428b-adb0-934029507ff6 req-037f1183-de42-4930-9ea4-eb92ee09844a service nova] Lock "9a705900-a0de-46e3-9af4-d07911f181c0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:54:39 user nova-compute[72081]: DEBUG nova.compute.manager [req-10167fe7-c96d-428b-adb0-934029507ff6 req-037f1183-de42-4930-9ea4-eb92ee09844a service nova] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] No waiting events found dispatching network-vif-plugged-55b4c29c-432e-4323-a5d6-fc62a42f8655 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:54:39 user nova-compute[72081]: WARNING nova.compute.manager [req-10167fe7-c96d-428b-adb0-934029507ff6 req-037f1183-de42-4930-9ea4-eb92ee09844a service nova] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Received unexpected event network-vif-plugged-55b4c29c-432e-4323-a5d6-fc62a42f8655 for instance with vm_state deleted and task_state None.
Apr 12 03:54:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:54:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:54:52 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681271677.7603638, 9a705900-a0de-46e3-9af4-d07911f181c0 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:54:52 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] VM Stopped (Lifecycle Event)
Apr 12 03:54:52 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:54:52 user nova-compute[72081]: DEBUG nova.compute.manager [None req-a78bef70-9f79-4c2a-8b30-1c658295237f None None] [instance: 9a705900-a0de-46e3-9af4-d07911f181c0] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:54:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:55:01 user nova-compute[72081]: DEBUG nova.compute.manager [req-1b9b9154-d440-4a4e-ad31-505ed60fa8aa req-aec2aeb0-6778-4964-9406-96f1494b7561 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Received event network-changed-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:55:01 user nova-compute[72081]: DEBUG nova.compute.manager [req-1b9b9154-d440-4a4e-ad31-505ed60fa8aa req-aec2aeb0-6778-4964-9406-96f1494b7561 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Refreshing instance network info cache due to event network-changed-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:55:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b9b9154-d440-4a4e-ad31-505ed60fa8aa req-aec2aeb0-6778-4964-9406-96f1494b7561 service nova] Acquiring lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:55:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b9b9154-d440-4a4e-ad31-505ed60fa8aa req-aec2aeb0-6778-4964-9406-96f1494b7561 service nova] Acquired lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:55:01 user nova-compute[72081]: DEBUG nova.network.neutron [req-1b9b9154-d440-4a4e-ad31-505ed60fa8aa req-aec2aeb0-6778-4964-9406-96f1494b7561 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Refreshing network info cache for port 1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:55:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Acquiring lock "1e4dae15-f69f-4e44-959a-7480e185020b" by "nova.compute.manager.ComputeManager.shelve_offload_instance.<locals>.do_shelve_offload_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:55:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "1e4dae15-f69f-4e44-959a-7480e185020b" acquired by "nova.compute.manager.ComputeManager.shelve_offload_instance.<locals>.do_shelve_offload_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:55:02 user nova-compute[72081]: INFO nova.compute.manager [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Shelve offloading
Apr 12 03:55:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:55:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:55:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:55:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:55:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:55:02 user nova-compute[72081]: DEBUG nova.network.neutron [req-1b9b9154-d440-4a4e-ad31-505ed60fa8aa req-aec2aeb0-6778-4964-9406-96f1494b7561 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Updated VIF entry in instance network info cache for port 1be56e0c-9ee5-4df6-bbd6-f866a4e54e90. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:55:02 user nova-compute[72081]: DEBUG nova.network.neutron [req-1b9b9154-d440-4a4e-ad31-505ed60fa8aa req-aec2aeb0-6778-4964-9406-96f1494b7561 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Updating instance_info_cache with network_info: [{"id": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "address": "fa:16:3e:e9:ff:d5", "network": {"id": "99c0b9a9-e1b4-43d1-bbbb-66560eca330a", "bridge": "br-int", "label": "tempest-TestShelveInstance-2092145606-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.62", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d8ec2318ea05403db924630461ea77e5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be56e0c-9e", "ovs_interfaceid": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:55:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b9b9154-d440-4a4e-ad31-505ed60fa8aa req-aec2aeb0-6778-4964-9406-96f1494b7561 service nova] Releasing lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:55:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:55:02 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Instance destroyed successfully.
Apr 12 03:55:02 user nova-compute[72081]: DEBUG nova.compute.manager [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:55:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Acquiring lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:55:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Acquired lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:55:02 user nova-compute[72081]: DEBUG nova.network.neutron [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:55:03 user nova-compute[72081]: DEBUG nova.network.neutron [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Updating instance_info_cache with network_info: [{"id": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "address": "fa:16:3e:e9:ff:d5", "network": {"id": "99c0b9a9-e1b4-43d1-bbbb-66560eca330a", "bridge": "br-int", "label": "tempest-TestShelveInstance-2092145606-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.62", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d8ec2318ea05403db924630461ea77e5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be56e0c-9e", "ovs_interfaceid": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:55:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Releasing lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:55:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:55:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-4aa270ca-aa5d-4113-8b36-5ee5e601073e req-18c1ab42-3add-49c5-ba3d-ed1b3b6379d0 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Received event network-vif-unplugged-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:55:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4aa270ca-aa5d-4113-8b36-5ee5e601073e req-18c1ab42-3add-49c5-ba3d-ed1b3b6379d0 service nova] Acquiring lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:55:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4aa270ca-aa5d-4113-8b36-5ee5e601073e req-18c1ab42-3add-49c5-ba3d-ed1b3b6379d0 service nova] Lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:55:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4aa270ca-aa5d-4113-8b36-5ee5e601073e req-18c1ab42-3add-49c5-ba3d-ed1b3b6379d0 service nova] Lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:55:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-4aa270ca-aa5d-4113-8b36-5ee5e601073e req-18c1ab42-3add-49c5-ba3d-ed1b3b6379d0 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] No waiting events found dispatching network-vif-unplugged-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:55:03 user nova-compute[72081]: WARNING nova.compute.manager [req-4aa270ca-aa5d-4113-8b36-5ee5e601073e req-18c1ab42-3add-49c5-ba3d-ed1b3b6379d0 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Received unexpected event network-vif-unplugged-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 for instance with vm_state active and task_state shelving.
Apr 12 03:55:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-4aa270ca-aa5d-4113-8b36-5ee5e601073e req-18c1ab42-3add-49c5-ba3d-ed1b3b6379d0 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Received event network-vif-plugged-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:55:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4aa270ca-aa5d-4113-8b36-5ee5e601073e req-18c1ab42-3add-49c5-ba3d-ed1b3b6379d0 service nova] Acquiring lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:55:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4aa270ca-aa5d-4113-8b36-5ee5e601073e req-18c1ab42-3add-49c5-ba3d-ed1b3b6379d0 service nova] Lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:55:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4aa270ca-aa5d-4113-8b36-5ee5e601073e req-18c1ab42-3add-49c5-ba3d-ed1b3b6379d0 service nova] Lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:55:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-4aa270ca-aa5d-4113-8b36-5ee5e601073e req-18c1ab42-3add-49c5-ba3d-ed1b3b6379d0 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] No waiting events found dispatching network-vif-plugged-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:55:03 user nova-compute[72081]: WARNING nova.compute.manager [req-4aa270ca-aa5d-4113-8b36-5ee5e601073e req-18c1ab42-3add-49c5-ba3d-ed1b3b6379d0 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Received unexpected event network-vif-plugged-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 for instance with vm_state active and task_state shelving.
Apr 12 03:55:04 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Instance destroyed successfully.
Apr 12 03:55:04 user nova-compute[72081]: DEBUG nova.objects.instance [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lazy-loading 'resources' on Instance uuid 1e4dae15-f69f-4e44-959a-7480e185020b {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:53:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestShelveInstance-server-1000809041',display_name='tempest-TestShelveInstance-server-1000809041',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testshelveinstance-server-1000809041',id=42,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHHBsGFyn1IfBui/0BXjt7hpTkTH4PsZRTzQ4VM6GFP5pgRYv1Yo7P5WOmhK4SKKnd1P8T0z7p8gKrx3Yxp/sOlxdxioj+WGXT88SaOKal87PVel/g0WojZ8NRP1p7DX/g==',key_name='tempest-TestShelveInstance-1986276884',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:53:21Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='d8ec2318ea05403db924630461ea77e5',ramdisk_id='',reservation_id='r-3erg1hz6',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestShelveInstance-130328817',owner_user_name='tempest-TestShelveInstance-130328817-project-member'},tags=<?>,task_state='shelving',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:53:21Z,user_data=None,user_id='447437f9d71b4f8ab0da168f0fca732f',uuid=1e4dae15-f69f-4e44-959a-7480e185020b,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "address": "fa:16:3e:e9:ff:d5", "network": {"id": "99c0b9a9-e1b4-43d1-bbbb-66560eca330a", "bridge": "br-int", "label": "tempest-TestShelveInstance-2092145606-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.62", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d8ec2318ea05403db924630461ea77e5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be56e0c-9e", "ovs_interfaceid": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Converting VIF {"id": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "address": "fa:16:3e:e9:ff:d5", "network": {"id": "99c0b9a9-e1b4-43d1-bbbb-66560eca330a", "bridge": "br-int", "label": "tempest-TestShelveInstance-2092145606-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.62", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d8ec2318ea05403db924630461ea77e5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be56e0c-9e", "ovs_interfaceid": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e9:ff:d5,bridge_name='br-int',has_traffic_filtering=True,id=1be56e0c-9ee5-4df6-bbd6-f866a4e54e90,network=Network(99c0b9a9-e1b4-43d1-bbbb-66560eca330a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1be56e0c-9e') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG os_vif [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:e9:ff:d5,bridge_name='br-int',has_traffic_filtering=True,id=1be56e0c-9ee5-4df6-bbd6-f866a4e54e90,network=Network(99c0b9a9-e1b4-43d1-bbbb-66560eca330a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1be56e0c-9e') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap1be56e0c-9e, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:55:04 user nova-compute[72081]: INFO os_vif [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:e9:ff:d5,bridge_name='br-int',has_traffic_filtering=True,id=1be56e0c-9ee5-4df6-bbd6-f866a4e54e90,network=Network(99c0b9a9-e1b4-43d1-bbbb-66560eca330a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1be56e0c-9e')
Apr 12 03:55:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OZAguYvD', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:55:04 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:55:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-1499c205-2681-470d-a576-f14d45e6b3fb req-5b49bc36-9a8f-4df1-86fb-186b7025c3e3 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Received event network-changed-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-1499c205-2681-470d-a576-f14d45e6b3fb req-5b49bc36-9a8f-4df1-86fb-186b7025c3e3 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Refreshing instance network info cache due to event network-changed-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1499c205-2681-470d-a576-f14d45e6b3fb req-5b49bc36-9a8f-4df1-86fb-186b7025c3e3 service nova] Acquiring lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1499c205-2681-470d-a576-f14d45e6b3fb req-5b49bc36-9a8f-4df1-86fb-186b7025c3e3 service nova] Acquired lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG nova.network.neutron [req-1499c205-2681-470d-a576-f14d45e6b3fb req-5b49bc36-9a8f-4df1-86fb-186b7025c3e3 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Refreshing network info cache for port 1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7b52e41a-a37b-416c-b287-f2da0be0e5da]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:55:04 user nova-compute[72081]: No targets found.
Apr 12 03:55:04 user nova-compute[72081]: iSNS:
Apr 12 03:55:04 user nova-compute[72081]: No targets found.
Apr 12 03:55:04 user nova-compute[72081]: STATIC:
Apr 12 03:55:04 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd
Apr 12 03:55:04 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:55:04 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:55:04 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02
Apr 12 03:55:04 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:55:04 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:55:04 user nova-compute[72081]: FIRMWARE:
Apr 12 03:55:04 user nova-compute[72081]: No targets found.
Apr 12 03:55:04 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:55:04 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:55:04 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[70911ee8-81bf-4f99-9f47-58a5c45f511c]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[590ab196-307d-4433-9241-bb79ef0fcc30]: (4, ('tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:55:04 user nova-compute[72081]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)
Apr 12 03:55:04 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsi session list stdout=tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:55:04 user nova-compute[72081]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)
Apr 12 03:55:04 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fac2ccca0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02'): ({'sdc'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Removing single pathed devices sdc {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[964c8c24-36fd-490e-842e-18d684936ebc]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b907312-dbce-4d4a-8062-6adf62ad8646]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Flushing IO for device /dev/sdc {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6f90ed65-0ebb-4765-a6b7-abba8437ce90]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.055s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[cbe3f123-ec18-45e3-8aa8-97cf09ddad4d]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] SCSI volumes sdc have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[273d879e-3516-4c6b-b905-73dc773bc16b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --logout" returned: 0 in 0.083s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5cd20029-eea1-4c67-aa96-6ac3be91ca78]: (4, ('Logging out of session [sid: 46, target: iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02, portal: 172.16.0.220,3260]\nLogout of [sid: 46, target: iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 46, target: iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02, portal: 172.16.0.220,3260]
Apr 12 03:55:04 user nova-compute[72081]: Logout of [sid: 46, target: iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02, portal: 172.16.0.220,3260] successful.
Apr 12 03:55:04 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[96d06c0e-ef05-455e-a60b-e9c9e151f960]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.338s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] <== disconnect_volume: return (339ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:55:04 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Deleting instance files /opt/stack/data/nova/instances/1e4dae15-f69f-4e44-959a-7480e185020b_del
Apr 12 03:55:04 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Deletion of /opt/stack/data/nova/instances/1e4dae15-f69f-4e44-959a-7480e185020b_del complete
Apr 12 03:55:04 user nova-compute[72081]: DEBUG nova.network.neutron [req-1499c205-2681-470d-a576-f14d45e6b3fb req-5b49bc36-9a8f-4df1-86fb-186b7025c3e3 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Updated VIF entry in instance network info cache for port 1be56e0c-9ee5-4df6-bbd6-f866a4e54e90. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG nova.network.neutron [req-1499c205-2681-470d-a576-f14d45e6b3fb req-5b49bc36-9a8f-4df1-86fb-186b7025c3e3 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Updating instance_info_cache with network_info: [{"id": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "address": "fa:16:3e:e9:ff:d5", "network": {"id": "99c0b9a9-e1b4-43d1-bbbb-66560eca330a", "bridge": null, "label": "tempest-TestShelveInstance-2092145606-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.62", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d8ec2318ea05403db924630461ea77e5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "unbound", "details": {}, "devname": "tap1be56e0c-9e", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:55:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1499c205-2681-470d-a576-f14d45e6b3fb req-5b49bc36-9a8f-4df1-86fb-186b7025c3e3 service nova] Releasing lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:55:05 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Deleted allocations for instance 1e4dae15-f69f-4e44-959a-7480e185020b
Apr 12 03:55:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:55:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:55:05 user nova-compute[72081]: DEBUG nova.objects.instance [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lazy-loading 'numa_topology' on Instance uuid 1e4dae15-f69f-4e44-959a-7480e185020b {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:55:05 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:55:05 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:55:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.138s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:55:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-51c1e833-e109-49de-9fd1-e101a1d9e1ad tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "1e4dae15-f69f-4e44-959a-7480e185020b" "released" by "nova.compute.manager.ComputeManager.shelve_offload_instance.<locals>.do_shelve_offload_instance" :: held 3.632s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:55:08 user nova-compute[72081]: INFO nova.compute.manager [None req-7d608da2-fac9-47cc-82df-8d3e1d85266c tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Get console output
Apr 12 03:55:08 user nova-compute[72081]: INFO nova.privsep.libvirt [-] Ignored error while reading from instance console pty: can't concat NoneType to bytes
Apr 12 03:55:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:55:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:55:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:55:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:55:17 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681271702.8046806, 1e4dae15-f69f-4e44-959a-7480e185020b => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:55:17 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] VM Stopped (Lifecycle Event)
Apr 12 03:55:17 user nova-compute[72081]: DEBUG nova.compute.manager [None req-730df0fa-6714-4f2a-9c0f-527b4827ad82 None None] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:55:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:55:18 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:55:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:55:19 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:55:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:55:24 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:55:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:55:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:55:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:55:24 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:55:24 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:55:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:55:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json" returned: 0 in 0.199s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:55:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:55:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json" returned: 0 in 0.171s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:55:25 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:55:25 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:55:25 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8648MB free_disk=26.052356719970703GB free_vcpus=10 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:55:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:55:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:55:25 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:55:25 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 65ba7373-1427-409d-9aa7-6cb7894d665b actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 192, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:55:25 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 2 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:55:25 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=832MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=2 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:55:25 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:55:25 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:55:25 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:55:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.235s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:55:26 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:55:26 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:55:26 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:55:26 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:55:26 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:55:28 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:55:28 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:55:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-65ba7373-1427-409d-9aa7-6cb7894d665b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:55:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-65ba7373-1427-409d-9aa7-6cb7894d665b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:55:28 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 03:55:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:55:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:55:29 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Updating instance_info_cache with network_info: [{"id": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "address": "fa:16:3e:e9:20:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e82a6e-18", "ovs_interfaceid": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:55:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-65ba7373-1427-409d-9aa7-6cb7894d665b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:55:29 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 03:55:29 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:55:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:55:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:55:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:55:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:55:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:55:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:55:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 03:55:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 03:55:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 03:55:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:55:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:55:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:55:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 03:55:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 03:55:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 03:55:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:55:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:55:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:56:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Acquiring lock "faa9190f-a85c-4107-9c7c-3de959b5d4d0" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:56:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Lock "faa9190f-a85c-4107-9c7c-3de959b5d4d0" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:56:01 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:56:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:56:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:56:01 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:56:01 user nova-compute[72081]: INFO nova.compute.claims [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Claim successful on node user
Apr 12 03:56:02 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:56:02 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:56:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.269s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:56:02 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:56:02 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:56:02 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:56:02 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:56:02 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:56:02 user nova-compute[72081]: DEBUG nova.policy [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '79bd18d6965c4f85bbbc121864fe5782', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4b1260a71aa436081e636ca7b996ead', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:56:02 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:56:02 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:56:02 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Creating image(s)
Apr 12 03:56:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Acquiring lock "/opt/stack/data/nova/instances/faa9190f-a85c-4107-9c7c-3de959b5d4d0/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:56:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Lock "/opt/stack/data/nova/instances/faa9190f-a85c-4107-9c7c-3de959b5d4d0/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:56:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Lock "/opt/stack/data/nova/instances/faa9190f-a85c-4107-9c7c-3de959b5d4d0/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:56:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.147s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:56:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:56:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:56:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.131s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:56:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/faa9190f-a85c-4107-9c7c-3de959b5d4d0/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/faa9190f-a85c-4107-9c7c-3de959b5d4d0/disk 1073741824" returned: 0 in 0.054s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:56:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.192s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:56:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.166s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:56:02 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Checking if we can resize image /opt/stack/data/nova/instances/faa9190f-a85c-4107-9c7c-3de959b5d4d0/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:56:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/faa9190f-a85c-4107-9c7c-3de959b5d4d0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:03 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Successfully created port: 8531b3ef-8643-4221-8728-14adc021e84f {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:56:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/faa9190f-a85c-4107-9c7c-3de959b5d4d0/disk --force-share --output=json" returned: 0 in 0.150s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:56:03 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Cannot resize image /opt/stack/data/nova/instances/faa9190f-a85c-4107-9c7c-3de959b5d4d0/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:56:03 user nova-compute[72081]: DEBUG nova.objects.instance [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Lazy-loading 'migration_context' on Instance uuid faa9190f-a85c-4107-9c7c-3de959b5d4d0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:56:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:56:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Ensure instance console log exists: /opt/stack/data/nova/instances/faa9190f-a85c-4107-9c7c-3de959b5d4d0/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:56:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:56:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:56:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:56:03 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Successfully updated port: 8531b3ef-8643-4221-8728-14adc021e84f {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:56:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Acquiring lock "refresh_cache-faa9190f-a85c-4107-9c7c-3de959b5d4d0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:56:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Acquired lock "refresh_cache-faa9190f-a85c-4107-9c7c-3de959b5d4d0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:56:03 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:56:03 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:56:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-312b2dc4-ba44-4ef2-b85d-9a309a8626aa req-b1b9c507-8c81-4452-a1cd-b876f98ed978 service nova] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Received event network-changed-8531b3ef-8643-4221-8728-14adc021e84f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:56:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-312b2dc4-ba44-4ef2-b85d-9a309a8626aa req-b1b9c507-8c81-4452-a1cd-b876f98ed978 service nova] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Refreshing instance network info cache due to event network-changed-8531b3ef-8643-4221-8728-14adc021e84f. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:56:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-312b2dc4-ba44-4ef2-b85d-9a309a8626aa req-b1b9c507-8c81-4452-a1cd-b876f98ed978 service nova] Acquiring lock "refresh_cache-faa9190f-a85c-4107-9c7c-3de959b5d4d0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Updating instance_info_cache with network_info: [{"id": "8531b3ef-8643-4221-8728-14adc021e84f", "address": "fa:16:3e:0d:2b:d9", "network": {"id": "123991b1-68ee-4b1b-8ce9-35b830ec5822", "bridge": "br-int", "label": "tempest-VolumesActionsTest-820420474-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "b4b1260a71aa436081e636ca7b996ead", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8531b3ef-86", "ovs_interfaceid": "8531b3ef-8643-4221-8728-14adc021e84f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Releasing lock "refresh_cache-faa9190f-a85c-4107-9c7c-3de959b5d4d0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Instance network_info: |[{"id": "8531b3ef-8643-4221-8728-14adc021e84f", "address": "fa:16:3e:0d:2b:d9", "network": {"id": "123991b1-68ee-4b1b-8ce9-35b830ec5822", "bridge": "br-int", "label": "tempest-VolumesActionsTest-820420474-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "b4b1260a71aa436081e636ca7b996ead", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8531b3ef-86", "ovs_interfaceid": "8531b3ef-8643-4221-8728-14adc021e84f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-312b2dc4-ba44-4ef2-b85d-9a309a8626aa req-b1b9c507-8c81-4452-a1cd-b876f98ed978 service nova] Acquired lock "refresh_cache-faa9190f-a85c-4107-9c7c-3de959b5d4d0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.network.neutron [req-312b2dc4-ba44-4ef2-b85d-9a309a8626aa req-b1b9c507-8c81-4452-a1cd-b876f98ed978 service nova] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Refreshing network info cache for port 8531b3ef-8643-4221-8728-14adc021e84f {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Start _get_guest_xml network_info=[{"id": "8531b3ef-8643-4221-8728-14adc021e84f", "address": "fa:16:3e:0d:2b:d9", "network": {"id": "123991b1-68ee-4b1b-8ce9-35b830ec5822", "bridge": "br-int", "label": "tempest-VolumesActionsTest-820420474-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "b4b1260a71aa436081e636ca7b996ead", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8531b3ef-86", "ovs_interfaceid": "8531b3ef-8643-4221-8728-14adc021e84f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:56:04 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:56:04 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:56:01Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-480847617',display_name='tempest-VolumesActionsTest-instance-480847617',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-480847617',id=43,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='b4b1260a71aa436081e636ca7b996ead',ramdisk_id='',reservation_id='r-y7vjjd74',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-437031612',owner_user_name='tempest-VolumesActionsTest-437031612-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:56:02Z,user_data=None,user_id='79bd18d6965c4f85bbbc121864fe5782',uuid=faa9190f-a85c-4107-9c7c-3de959b5d4d0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8531b3ef-8643-4221-8728-14adc021e84f", "address": "fa:16:3e:0d:2b:d9", "network": {"id": "123991b1-68ee-4b1b-8ce9-35b830ec5822", "bridge": "br-int", "label": "tempest-VolumesActionsTest-820420474-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "b4b1260a71aa436081e636ca7b996ead", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8531b3ef-86", "ovs_interfaceid": "8531b3ef-8643-4221-8728-14adc021e84f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Converting VIF {"id": "8531b3ef-8643-4221-8728-14adc021e84f", "address": "fa:16:3e:0d:2b:d9", "network": {"id": "123991b1-68ee-4b1b-8ce9-35b830ec5822", "bridge": "br-int", "label": "tempest-VolumesActionsTest-820420474-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "b4b1260a71aa436081e636ca7b996ead", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8531b3ef-86", "ovs_interfaceid": "8531b3ef-8643-4221-8728-14adc021e84f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:0d:2b:d9,bridge_name='br-int',has_traffic_filtering=True,id=8531b3ef-8643-4221-8728-14adc021e84f,network=Network(123991b1-68ee-4b1b-8ce9-35b830ec5822),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8531b3ef-86') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.objects.instance [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Lazy-loading 'pci_devices' on Instance uuid faa9190f-a85c-4107-9c7c-3de959b5d4d0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:56:04 user nova-compute[72081]:   <uuid>faa9190f-a85c-4107-9c7c-3de959b5d4d0</uuid>
Apr 12 03:56:04 user nova-compute[72081]:   <name>instance-0000002b</name>
Apr 12 03:56:04 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:56:04 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:56:04 user nova-compute[72081]:   <metadata>
Apr 12 03:56:04 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:56:04 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:56:04 user nova-compute[72081]:       <nova:name>tempest-VolumesActionsTest-instance-480847617</nova:name>
Apr 12 03:56:04 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:56:04</nova:creationTime>
Apr 12 03:56:04 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:56:04 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:56:04 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:56:04 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:56:04 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:56:04 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:56:04 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:56:04 user nova-compute[72081]:       <nova:owner>
Apr 12 03:56:04 user nova-compute[72081]:         <nova:user uuid="79bd18d6965c4f85bbbc121864fe5782">tempest-VolumesActionsTest-437031612-project-member</nova:user>
Apr 12 03:56:04 user nova-compute[72081]:         <nova:project uuid="b4b1260a71aa436081e636ca7b996ead">tempest-VolumesActionsTest-437031612</nova:project>
Apr 12 03:56:04 user nova-compute[72081]:       </nova:owner>
Apr 12 03:56:04 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:56:04 user nova-compute[72081]:       <nova:ports>
Apr 12 03:56:04 user nova-compute[72081]:         <nova:port uuid="8531b3ef-8643-4221-8728-14adc021e84f">
Apr 12 03:56:04 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 12 03:56:04 user nova-compute[72081]:         </nova:port>
Apr 12 03:56:04 user nova-compute[72081]:       </nova:ports>
Apr 12 03:56:04 user nova-compute[72081]:     </nova:instance>
Apr 12 03:56:04 user nova-compute[72081]:   </metadata>
Apr 12 03:56:04 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:56:04 user nova-compute[72081]:     <system>
Apr 12 03:56:04 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:56:04 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:56:04 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:56:04 user nova-compute[72081]:       <entry name="serial">faa9190f-a85c-4107-9c7c-3de959b5d4d0</entry>
Apr 12 03:56:04 user nova-compute[72081]:       <entry name="uuid">faa9190f-a85c-4107-9c7c-3de959b5d4d0</entry>
Apr 12 03:56:04 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:56:04 user nova-compute[72081]:     </system>
Apr 12 03:56:04 user nova-compute[72081]:   </sysinfo>
Apr 12 03:56:04 user nova-compute[72081]:   <os>
Apr 12 03:56:04 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:56:04 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:56:04 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:56:04 user nova-compute[72081]:   </os>
Apr 12 03:56:04 user nova-compute[72081]:   <features>
Apr 12 03:56:04 user nova-compute[72081]:     <acpi/>
Apr 12 03:56:04 user nova-compute[72081]:     <apic/>
Apr 12 03:56:04 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:56:04 user nova-compute[72081]:   </features>
Apr 12 03:56:04 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:56:04 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:56:04 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:56:04 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:56:04 user nova-compute[72081]:   </clock>
Apr 12 03:56:04 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:56:04 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:56:04 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:56:04 user nova-compute[72081]:   </cpu>
Apr 12 03:56:04 user nova-compute[72081]:   <devices>
Apr 12 03:56:04 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:56:04 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:56:04 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/faa9190f-a85c-4107-9c7c-3de959b5d4d0/disk"/>
Apr 12 03:56:04 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:56:04 user nova-compute[72081]:     </disk>
Apr 12 03:56:04 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:56:04 user nova-compute[72081]:       <mac address="fa:16:3e:0d:2b:d9"/>
Apr 12 03:56:04 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:56:04 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:56:04 user nova-compute[72081]:       <target dev="tap8531b3ef-86"/>
Apr 12 03:56:04 user nova-compute[72081]:     </interface>
Apr 12 03:56:04 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:56:04 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/faa9190f-a85c-4107-9c7c-3de959b5d4d0/console.log" append="off"/>
Apr 12 03:56:04 user nova-compute[72081]:     </serial>
Apr 12 03:56:04 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:56:04 user nova-compute[72081]:     <video>
Apr 12 03:56:04 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:56:04 user nova-compute[72081]:     </video>
Apr 12 03:56:04 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:56:04 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:56:04 user nova-compute[72081]:     </rng>
Apr 12 03:56:04 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:56:04 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:56:04 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:56:04 user nova-compute[72081]:     </memballoon>
Apr 12 03:56:04 user nova-compute[72081]:   </devices>
Apr 12 03:56:04 user nova-compute[72081]: </domain>
Apr 12 03:56:04 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:56:01Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-480847617',display_name='tempest-VolumesActionsTest-instance-480847617',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-480847617',id=43,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='b4b1260a71aa436081e636ca7b996ead',ramdisk_id='',reservation_id='r-y7vjjd74',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-437031612',owner_user_name='tempest-VolumesActionsTest-437031612-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:56:02Z,user_data=None,user_id='79bd18d6965c4f85bbbc121864fe5782',uuid=faa9190f-a85c-4107-9c7c-3de959b5d4d0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8531b3ef-8643-4221-8728-14adc021e84f", "address": "fa:16:3e:0d:2b:d9", "network": {"id": "123991b1-68ee-4b1b-8ce9-35b830ec5822", "bridge": "br-int", "label": "tempest-VolumesActionsTest-820420474-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "b4b1260a71aa436081e636ca7b996ead", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8531b3ef-86", "ovs_interfaceid": "8531b3ef-8643-4221-8728-14adc021e84f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Converting VIF {"id": "8531b3ef-8643-4221-8728-14adc021e84f", "address": "fa:16:3e:0d:2b:d9", "network": {"id": "123991b1-68ee-4b1b-8ce9-35b830ec5822", "bridge": "br-int", "label": "tempest-VolumesActionsTest-820420474-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "b4b1260a71aa436081e636ca7b996ead", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8531b3ef-86", "ovs_interfaceid": "8531b3ef-8643-4221-8728-14adc021e84f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:0d:2b:d9,bridge_name='br-int',has_traffic_filtering=True,id=8531b3ef-8643-4221-8728-14adc021e84f,network=Network(123991b1-68ee-4b1b-8ce9-35b830ec5822),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8531b3ef-86') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG os_vif [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:0d:2b:d9,bridge_name='br-int',has_traffic_filtering=True,id=8531b3ef-8643-4221-8728-14adc021e84f,network=Network(123991b1-68ee-4b1b-8ce9-35b830ec5822),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8531b3ef-86') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap8531b3ef-86, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap8531b3ef-86, col_values=(('external_ids', {'iface-id': '8531b3ef-8643-4221-8728-14adc021e84f', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:0d:2b:d9', 'vm-uuid': 'faa9190f-a85c-4107-9c7c-3de959b5d4d0'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:04 user nova-compute[72081]: INFO os_vif [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:0d:2b:d9,bridge_name='br-int',has_traffic_filtering=True,id=8531b3ef-8643-4221-8728-14adc021e84f,network=Network(123991b1-68ee-4b1b-8ce9-35b830ec5822),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8531b3ef-86')
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] No VIF found with MAC fa:16:3e:0d:2b:d9, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.network.neutron [req-312b2dc4-ba44-4ef2-b85d-9a309a8626aa req-b1b9c507-8c81-4452-a1cd-b876f98ed978 service nova] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Updated VIF entry in instance network info cache for port 8531b3ef-8643-4221-8728-14adc021e84f. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG nova.network.neutron [req-312b2dc4-ba44-4ef2-b85d-9a309a8626aa req-b1b9c507-8c81-4452-a1cd-b876f98ed978 service nova] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Updating instance_info_cache with network_info: [{"id": "8531b3ef-8643-4221-8728-14adc021e84f", "address": "fa:16:3e:0d:2b:d9", "network": {"id": "123991b1-68ee-4b1b-8ce9-35b830ec5822", "bridge": "br-int", "label": "tempest-VolumesActionsTest-820420474-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "b4b1260a71aa436081e636ca7b996ead", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8531b3ef-86", "ovs_interfaceid": "8531b3ef-8643-4221-8728-14adc021e84f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:56:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-312b2dc4-ba44-4ef2-b85d-9a309a8626aa req-b1b9c507-8c81-4452-a1cd-b876f98ed978 service nova] Releasing lock "refresh_cache-faa9190f-a85c-4107-9c7c-3de959b5d4d0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:56:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:05 user nova-compute[72081]: DEBUG nova.compute.manager [req-ded99285-1722-481a-bc2a-3c39a5f87626 req-73fdeedf-0d44-4400-ada2-e76a4e7a103f service nova] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Received event network-vif-plugged-8531b3ef-8643-4221-8728-14adc021e84f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:56:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ded99285-1722-481a-bc2a-3c39a5f87626 req-73fdeedf-0d44-4400-ada2-e76a4e7a103f service nova] Acquiring lock "faa9190f-a85c-4107-9c7c-3de959b5d4d0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:56:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ded99285-1722-481a-bc2a-3c39a5f87626 req-73fdeedf-0d44-4400-ada2-e76a4e7a103f service nova] Lock "faa9190f-a85c-4107-9c7c-3de959b5d4d0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:56:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ded99285-1722-481a-bc2a-3c39a5f87626 req-73fdeedf-0d44-4400-ada2-e76a4e7a103f service nova] Lock "faa9190f-a85c-4107-9c7c-3de959b5d4d0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:56:05 user nova-compute[72081]: DEBUG nova.compute.manager [req-ded99285-1722-481a-bc2a-3c39a5f87626 req-73fdeedf-0d44-4400-ada2-e76a4e7a103f service nova] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] No waiting events found dispatching network-vif-plugged-8531b3ef-8643-4221-8728-14adc021e84f {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:56:05 user nova-compute[72081]: WARNING nova.compute.manager [req-ded99285-1722-481a-bc2a-3c39a5f87626 req-73fdeedf-0d44-4400-ada2-e76a4e7a103f service nova] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Received unexpected event network-vif-plugged-8531b3ef-8643-4221-8728-14adc021e84f for instance with vm_state building and task_state spawning.
Apr 12 03:56:07 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271767.651762, faa9190f-a85c-4107-9c7c-3de959b5d4d0 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:56:07 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] VM Resumed (Lifecycle Event)
Apr 12 03:56:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:56:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:56:07 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Instance spawned successfully.
Apr 12 03:56:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:56:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:56:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:56:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:56:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:56:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:56:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:56:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:56:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:56:07 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:56:07 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271767.6563098, faa9190f-a85c-4107-9c7c-3de959b5d4d0 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:56:07 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] VM Started (Lifecycle Event)
Apr 12 03:56:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:56:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:56:07 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:56:07 user nova-compute[72081]: INFO nova.compute.manager [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Took 5.37 seconds to spawn the instance on the hypervisor.
Apr 12 03:56:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:56:07 user nova-compute[72081]: INFO nova.compute.manager [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Took 5.99 seconds to build instance.
Apr 12 03:56:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8a2499a4-67e1-4178-8d2a-cfa0979178ba tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Lock "faa9190f-a85c-4107-9c7c-3de959b5d4d0" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.084s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:56:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-1a03e474-99e7-4e80-8a06-d557c6458411 req-f461db22-f0e9-46e9-94eb-6fe685abe31e service nova] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Received event network-vif-plugged-8531b3ef-8643-4221-8728-14adc021e84f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:56:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a03e474-99e7-4e80-8a06-d557c6458411 req-f461db22-f0e9-46e9-94eb-6fe685abe31e service nova] Acquiring lock "faa9190f-a85c-4107-9c7c-3de959b5d4d0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:56:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a03e474-99e7-4e80-8a06-d557c6458411 req-f461db22-f0e9-46e9-94eb-6fe685abe31e service nova] Lock "faa9190f-a85c-4107-9c7c-3de959b5d4d0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:56:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1a03e474-99e7-4e80-8a06-d557c6458411 req-f461db22-f0e9-46e9-94eb-6fe685abe31e service nova] Lock "faa9190f-a85c-4107-9c7c-3de959b5d4d0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:56:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-1a03e474-99e7-4e80-8a06-d557c6458411 req-f461db22-f0e9-46e9-94eb-6fe685abe31e service nova] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] No waiting events found dispatching network-vif-plugged-8531b3ef-8643-4221-8728-14adc021e84f {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:56:07 user nova-compute[72081]: WARNING nova.compute.manager [req-1a03e474-99e7-4e80-8a06-d557c6458411 req-f461db22-f0e9-46e9-94eb-6fe685abe31e service nova] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Received unexpected event network-vif-plugged-8531b3ef-8643-4221-8728-14adc021e84f for instance with vm_state active and task_state None.
Apr 12 03:56:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "65ba7373-1427-409d-9aa7-6cb7894d665b" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:56:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:56:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:56:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:56:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:56:08 user nova-compute[72081]: INFO nova.compute.manager [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Terminating instance
Apr 12 03:56:08 user nova-compute[72081]: DEBUG nova.compute.manager [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:56:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-14336358-3f26-44b7-a0e0-c9922cf6f8f3 req-827b1ccd-a4c1-472a-9be3-fe3531deab45 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Received event network-vif-unplugged-d7e82a6e-18fe-44f2-b866-f0561d1b8209 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-14336358-3f26-44b7-a0e0-c9922cf6f8f3 req-827b1ccd-a4c1-472a-9be3-fe3531deab45 service nova] Acquiring lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-14336358-3f26-44b7-a0e0-c9922cf6f8f3 req-827b1ccd-a4c1-472a-9be3-fe3531deab45 service nova] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-14336358-3f26-44b7-a0e0-c9922cf6f8f3 req-827b1ccd-a4c1-472a-9be3-fe3531deab45 service nova] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-14336358-3f26-44b7-a0e0-c9922cf6f8f3 req-827b1ccd-a4c1-472a-9be3-fe3531deab45 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] No waiting events found dispatching network-vif-unplugged-d7e82a6e-18fe-44f2-b866-f0561d1b8209 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-14336358-3f26-44b7-a0e0-c9922cf6f8f3 req-827b1ccd-a4c1-472a-9be3-fe3531deab45 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Received event network-vif-unplugged-d7e82a6e-18fe-44f2-b866-f0561d1b8209 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:09 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Instance destroyed successfully.
Apr 12 03:56:09 user nova-compute[72081]: DEBUG nova.objects.instance [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'resources' on Instance uuid 65ba7373-1427-409d-9aa7-6cb7894d665b {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:49:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-441149863',display_name='tempest-ServerActionsTestJSON-server-441149863',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(12),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-441149863',id=40,image_ref='',info_cache=InstanceInfoCache,instance_type_id=12,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKmrI9wy1CUIwhlKuXvzk0mD7Pt5nZFvI1CsqS0do6Lk76JQOvGl5lvx9rkz7UAM9WGocGXyWSOUOuKTtYZRD2HqKUbTN4VPOoliE1yuu+s0lx7vuZu45Mj7wDb2S8Ksyw==',key_name='tempest-keypair-364276420',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:51:41Z,launched_on='user',locked=False,locked_by=None,memory_mb=192,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='815b65aa5ad9442db0a087c646f96bbd',ramdisk_id='',reservation_id='r-7tc52gi6',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestJSON-1898059859',owner_user_name='tempest-ServerActionsTestJSON-1898059859-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:53:19Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a7f422f1a05a4a92ba387e424850b360',uuid=65ba7373-1427-409d-9aa7-6cb7894d665b,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "address": "fa:16:3e:e9:20:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e82a6e-18", "ovs_interfaceid": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converting VIF {"id": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "address": "fa:16:3e:e9:20:ca", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e82a6e-18", "ovs_interfaceid": "d7e82a6e-18fe-44f2-b866-f0561d1b8209", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:e9:20:ca,bridge_name='br-int',has_traffic_filtering=True,id=d7e82a6e-18fe-44f2-b866-f0561d1b8209,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd7e82a6e-18') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG os_vif [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:e9:20:ca,bridge_name='br-int',has_traffic_filtering=True,id=d7e82a6e-18fe-44f2-b866-f0561d1b8209,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd7e82a6e-18') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapd7e82a6e-18, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:56:09 user nova-compute[72081]: INFO os_vif [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:e9:20:ca,bridge_name='br-int',has_traffic_filtering=True,id=d7e82a6e-18fe-44f2-b866-f0561d1b8209,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd7e82a6e-18')
Apr 12 03:56:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OVtlqBox', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:56:09 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:56:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[95c2a2ba-3841-420f-bc15-2fff577f2346]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:56:09 user nova-compute[72081]: No targets found.
Apr 12 03:56:09 user nova-compute[72081]: iSNS:
Apr 12 03:56:09 user nova-compute[72081]: No targets found.
Apr 12 03:56:09 user nova-compute[72081]: STATIC:
Apr 12 03:56:09 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd
Apr 12 03:56:09 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:56:09 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:56:09 user nova-compute[72081]: FIRMWARE:
Apr 12 03:56:09 user nova-compute[72081]: No targets found.
Apr 12 03:56:09 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:56:09 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:56:09 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3ba3e970-b58e-407a-a754-e85531c226f0]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[220cfb2a-6815-4991-b140-4f120a821c0f]: (4, ('tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:56:09 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsi session list stdout=tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd (non-flash)
Apr 12 03:56:09 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3f744029e0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd'): ({'sdb'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Removing single pathed devices sdb {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.029s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[cc328a83-d557-4302-bb38-9d654aa54370]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1dd08443-2bb2-4597-ae0b-c5d2e69b754f]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Flushing IO for device /dev/sdb {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.013s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1d4272f5-278c-442a-8fa5-534e0bf30c4f]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.060s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ef9c6bde-6f2a-4ccb-af80-12295f847ce2]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] SCSI volumes sdb have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-23736373732336531', '/dev/disk/by-id/scsi-SSCST_BIO_Storage_c38f22d1', '/dev/disk/by-id/scsi-1SCST_BIOc38f22d1-767723e180fd3f78') {{(pid=82083) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b2750081-7aa8-4a72-8b7e-d58424b0c196]: (4, None) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bddd1f06-a471-4110-8ee7-29a4f91e7431]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --logout" returned: 0 in 0.054s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9e6ac8a1-05cf-4850-b06d-042816c5c674]: (4, ('Logging out of session [sid: 44, target: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd, portal: 172.16.0.220,3260]\nLogout of [sid: 44, target: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 44, target: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd, portal: 172.16.0.220,3260]
Apr 12 03:56:09 user nova-compute[72081]: Logout of [sid: 44, target: iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd, portal: 172.16.0.220,3260] successful.
Apr 12 03:56:09 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f699cce1-7b7a-47fc-9c5e-161e74d0becd -p 172.16.0.220:3260 --op delete" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[60ea9fb1-c78b-4ca1-b281-2638734c9479]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.331s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] <== disconnect_volume: return (332ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:56:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:56:09 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Deleting instance files /opt/stack/data/nova/instances/65ba7373-1427-409d-9aa7-6cb7894d665b_del
Apr 12 03:56:09 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Deletion of /opt/stack/data/nova/instances/65ba7373-1427-409d-9aa7-6cb7894d665b_del complete
Apr 12 03:56:10 user nova-compute[72081]: INFO nova.compute.manager [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Took 1.24 seconds to destroy the instance on the hypervisor.
Apr 12 03:56:10 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:56:10 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:56:10 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:56:10 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:56:10 user nova-compute[72081]: DEBUG nova.compute.manager [req-aee2569e-7137-494e-8493-17fa48e0704e req-5f4972bc-6239-4f8e-bc15-dc20b0071a88 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Received event network-vif-deleted-d7e82a6e-18fe-44f2-b866-f0561d1b8209 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:56:10 user nova-compute[72081]: INFO nova.compute.manager [req-aee2569e-7137-494e-8493-17fa48e0704e req-5f4972bc-6239-4f8e-bc15-dc20b0071a88 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Neutron deleted interface d7e82a6e-18fe-44f2-b866-f0561d1b8209; detaching it from the instance and deleting it from the info cache
Apr 12 03:56:10 user nova-compute[72081]: DEBUG nova.network.neutron [req-aee2569e-7137-494e-8493-17fa48e0704e req-5f4972bc-6239-4f8e-bc15-dc20b0071a88 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:56:10 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Took 0.77 seconds to deallocate network for instance.
Apr 12 03:56:10 user nova-compute[72081]: DEBUG nova.compute.manager [req-aee2569e-7137-494e-8493-17fa48e0704e req-5f4972bc-6239-4f8e-bc15-dc20b0071a88 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Detach interface failed, port_id=d7e82a6e-18fe-44f2-b866-f0561d1b8209, reason: Instance 65ba7373-1427-409d-9aa7-6cb7894d665b could not be found. {{(pid=72081) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 12 03:56:11 user nova-compute[72081]: DEBUG nova.compute.manager [req-18847deb-7959-4062-b846-409274f28f93 req-42d4ea94-27d7-41b5-af3a-d0b1640b6b31 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Received event network-vif-plugged-d7e82a6e-18fe-44f2-b866-f0561d1b8209 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:56:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-18847deb-7959-4062-b846-409274f28f93 req-42d4ea94-27d7-41b5-af3a-d0b1640b6b31 service nova] Acquiring lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:56:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-18847deb-7959-4062-b846-409274f28f93 req-42d4ea94-27d7-41b5-af3a-d0b1640b6b31 service nova] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:56:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-18847deb-7959-4062-b846-409274f28f93 req-42d4ea94-27d7-41b5-af3a-d0b1640b6b31 service nova] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:56:11 user nova-compute[72081]: DEBUG nova.compute.manager [req-18847deb-7959-4062-b846-409274f28f93 req-42d4ea94-27d7-41b5-af3a-d0b1640b6b31 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] No waiting events found dispatching network-vif-plugged-d7e82a6e-18fe-44f2-b866-f0561d1b8209 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:56:11 user nova-compute[72081]: WARNING nova.compute.manager [req-18847deb-7959-4062-b846-409274f28f93 req-42d4ea94-27d7-41b5-af3a-d0b1640b6b31 service nova] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Received unexpected event network-vif-plugged-d7e82a6e-18fe-44f2-b866-f0561d1b8209 for instance with vm_state active and task_state deleting.
Apr 12 03:56:11 user nova-compute[72081]: INFO nova.compute.manager [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Took 0.84 seconds to detach 1 volumes for instance.
Apr 12 03:56:11 user nova-compute[72081]: DEBUG nova.compute.manager [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Deleting volume: f699cce1-7b7a-47fc-9c5e-161e74d0becd {{(pid=72081) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3202}}
Apr 12 03:56:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:56:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:56:11 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:56:11 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:56:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.209s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:56:12 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Deleted allocations for instance 65ba7373-1427-409d-9aa7-6cb7894d665b
Apr 12 03:56:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-edcbf72c-6cec-4922-9011-50d47b454a77 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "65ba7373-1427-409d-9aa7-6cb7894d665b" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.334s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:56:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:56:19 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:56:20 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:56:20 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:56:21 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:56:21 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Cleaning up deleted instances with incomplete migration  {{(pid=72081) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 12 03:56:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:24 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681271769.5918972, 65ba7373-1427-409d-9aa7-6cb7894d665b => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:56:24 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] VM Stopped (Lifecycle Event)
Apr 12 03:56:24 user nova-compute[72081]: DEBUG nova.compute.manager [None req-ae1f8bd2-5237-4006-8014-f3b4bb60ed9f None None] [instance: 65ba7373-1427-409d-9aa7-6cb7894d665b] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:56:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:24 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:56:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:56:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:56:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:56:24 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:56:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/faa9190f-a85c-4107-9c7c-3de959b5d4d0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/faa9190f-a85c-4107-9c7c-3de959b5d4d0/disk --force-share --output=json" returned: 0 in 0.164s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:56:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/faa9190f-a85c-4107-9c7c-3de959b5d4d0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/faa9190f-a85c-4107-9c7c-3de959b5d4d0/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:56:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:56:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:56:25 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:56:25 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:56:25 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8584MB free_disk=26.022480010986328GB free_vcpus=10 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:56:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:56:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:56:26 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:56:26 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance faa9190f-a85c-4107-9c7c-3de959b5d4d0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:56:26 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 2 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:56:26 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=768MB phys_disk=40GB used_disk=2GB total_vcpus=12 used_vcpus=2 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:56:26 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Refreshing inventories for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 12 03:56:26 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Updating ProviderTree inventory for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 12 03:56:26 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Updating inventory in ProviderTree for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 12 03:56:26 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Refreshing aggregate associations for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6, aggregates: None {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 12 03:56:26 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Refreshing trait associations for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6, traits: COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_USB,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_RESCUE_BFV,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE41,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_STORAGE_BUS_FDC,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_STORAGE_BUS_IDE,COMPUTE_TRUSTED_CERTS,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_GRAPHICS_MODEL_CIRRUS,HW_CPU_X86_SSSE3,HW_CPU_X86_SSE,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_EXTEND,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_STORAGE_BUS_SATA,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_GRAPHICS_MODEL_QXL,HW_CPU_X86_SSE42,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_ACCELERATORS,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,HW_CPU_X86_MMX,COMPUTE_NODE,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_DEVICE_TAGGING,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_VIOMMU_MODEL_AUTO {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 12 03:56:26 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:56:26 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:56:26 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:56:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.462s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:56:28 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:56:28 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:56:28 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:56:28 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:56:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:28 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:56:29 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:56:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:30 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:56:30 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:56:30 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Skipping network cache update for instance because it has been migrated to another host. {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9836}}
Apr 12 03:56:30 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Didn't find any instances for network info cache update. {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 12 03:56:30 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:56:30 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Cleaning up deleted instances {{(pid=72081) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 12 03:56:30 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] There are 0 instances to clean {{(pid=72081) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 12 03:56:31 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:56:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:56:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:50 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._cleanup_running_deleted_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:56:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Acquiring lock "1e4dae15-f69f-4e44-959a-7480e185020b" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:56:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "1e4dae15-f69f-4e44-959a-7480e185020b" acquired by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:56:53 user nova-compute[72081]: INFO nova.compute.manager [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Unshelving
Apr 12 03:56:53 user nova-compute[72081]: INFO nova.virt.block_device [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Booting with volume d6a07e3a-be2a-4191-b1b7-09667064fb02 at /dev/vda
Apr 12 03:56:53 user nova-compute[72081]: DEBUG os_brick.utils [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:56:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:56:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ece0a1ea-111d-4c1e-9667-4262c05d4ee0]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:56:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:56:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c0024df3-8199-4090-bb1d-479d0ee18d54]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:56:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:56:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[05bcf1b2-e881-4da4-b1f3-27d1eb28671f]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:56:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c2cc2210-ab9b-4d0c-9a59-e7b7cc033612]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:56:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:56:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 03:56:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 03:56:53 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 03:56:53 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c725fb7-5e30-4ace-bde3-9797b14690c4]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:56:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 03:56:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 03:56:53 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 03:56:53 user nova-compute[72081]: DEBUG os_brick.utils [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] <== get_connector_properties: return (138ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:56:53 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Updating existing volume attachment record: 478c8b84-aae3-4785-b9b0-79b45ee6acad {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 03:56:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:56:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:56:57 user nova-compute[72081]: DEBUG nova.objects.instance [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lazy-loading 'pci_requests' on Instance uuid 1e4dae15-f69f-4e44-959a-7480e185020b {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:56:57 user nova-compute[72081]: DEBUG nova.objects.instance [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lazy-loading 'numa_topology' on Instance uuid 1e4dae15-f69f-4e44-959a-7480e185020b {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:56:57 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:56:57 user nova-compute[72081]: INFO nova.compute.claims [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Claim successful on node user
Apr 12 03:56:57 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:56:57 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:56:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.248s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:56:58 user nova-compute[72081]: INFO nova.network.neutron [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Updating port 1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 with attributes {'binding:host_id': 'user', 'device_owner': 'compute:nova'}
Apr 12 03:56:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Acquiring lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Acquired lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.network.neutron [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-e3aac771-43df-431e-9a6d-77b3e718794a req-5b75bdfb-f198-489f-91bd-cb3e298f7db0 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Received event network-changed-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-e3aac771-43df-431e-9a6d-77b3e718794a req-5b75bdfb-f198-489f-91bd-cb3e298f7db0 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Refreshing instance network info cache due to event network-changed-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e3aac771-43df-431e-9a6d-77b3e718794a req-5b75bdfb-f198-489f-91bd-cb3e298f7db0 service nova] Acquiring lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7ea64428-ef2b-458e-91e6-05fec33cdba3 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "8d533f5f-dbed-4878-a9b1-2ce4fae8fa13" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7ea64428-ef2b-458e-91e6-05fec33cdba3 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "8d533f5f-dbed-4878-a9b1-2ce4fae8fa13" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7ea64428-ef2b-458e-91e6-05fec33cdba3 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "8d533f5f-dbed-4878-a9b1-2ce4fae8fa13-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7ea64428-ef2b-458e-91e6-05fec33cdba3 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "8d533f5f-dbed-4878-a9b1-2ce4fae8fa13-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7ea64428-ef2b-458e-91e6-05fec33cdba3 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "8d533f5f-dbed-4878-a9b1-2ce4fae8fa13-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:56:59 user nova-compute[72081]: INFO nova.compute.manager [None req-7ea64428-ef2b-458e-91e6-05fec33cdba3 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Terminating instance
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.compute.manager [None req-7ea64428-ef2b-458e-91e6-05fec33cdba3 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-9d892056-6b7a-426b-8683-9469e4a6b453 req-a7f3042b-aa7d-4166-8f9c-2839e07c1fba service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Received event network-vif-unplugged-2ef93926-6cf3-4d57-be98-ab555ac58b4c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9d892056-6b7a-426b-8683-9469e4a6b453 req-a7f3042b-aa7d-4166-8f9c-2839e07c1fba service nova] Acquiring lock "8d533f5f-dbed-4878-a9b1-2ce4fae8fa13-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9d892056-6b7a-426b-8683-9469e4a6b453 req-a7f3042b-aa7d-4166-8f9c-2839e07c1fba service nova] Lock "8d533f5f-dbed-4878-a9b1-2ce4fae8fa13-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9d892056-6b7a-426b-8683-9469e4a6b453 req-a7f3042b-aa7d-4166-8f9c-2839e07c1fba service nova] Lock "8d533f5f-dbed-4878-a9b1-2ce4fae8fa13-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-9d892056-6b7a-426b-8683-9469e4a6b453 req-a7f3042b-aa7d-4166-8f9c-2839e07c1fba service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] No waiting events found dispatching network-vif-unplugged-2ef93926-6cf3-4d57-be98-ab555ac58b4c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-9d892056-6b7a-426b-8683-9469e4a6b453 req-a7f3042b-aa7d-4166-8f9c-2839e07c1fba service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Received event network-vif-unplugged-2ef93926-6cf3-4d57-be98-ab555ac58b4c for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.network.neutron [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Updating instance_info_cache with network_info: [{"id": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "address": "fa:16:3e:e9:ff:d5", "network": {"id": "99c0b9a9-e1b4-43d1-bbbb-66560eca330a", "bridge": "br-int", "label": "tempest-TestShelveInstance-2092145606-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.62", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d8ec2318ea05403db924630461ea77e5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be56e0c-9e", "ovs_interfaceid": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Releasing lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:56:59 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Creating image(s)
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Did not create local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Ensure instance console log exists: /opt/stack/data/nova/instances/1e4dae15-f69f-4e44-959a-7480e185020b/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Start _get_guest_xml network_info=[{"id": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "address": "fa:16:3e:e9:ff:d5", "network": {"id": "99c0b9a9-e1b4-43d1-bbbb-66560eca330a", "bridge": "br-int", "label": "tempest-TestShelveInstance-2092145606-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.62", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d8ec2318ea05403db924630461ea77e5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be56e0c-9e", "ovs_interfaceid": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OZAguYvD', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '1e4dae15-f69f-4e44-959a-7480e185020b', 'attached_at': '', 'detached_at': '', 'volume_id': 'd6a07e3a-be2a-4191-b1b7-09667064fb02', 'serial': 'd6a07e3a-be2a-4191-b1b7-09667064fb02'}, 'disk_bus': 'virtio', 'delete_on_termination': True, 'boot_index': 0, 'guest_format': None, 'device_type': 'disk', 'attachment_id': '478c8b84-aae3-4785-b9b0-79b45ee6acad', 'volume_type': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e3aac771-43df-431e-9a6d-77b3e718794a req-5b75bdfb-f198-489f-91bd-cb3e298f7db0 service nova] Acquired lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.network.neutron [req-e3aac771-43df-431e-9a6d-77b3e718794a req-5b75bdfb-f198-489f-91bd-cb3e298f7db0 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Refreshing network info cache for port 1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:56:59 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:56:59 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.objects.instance [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lazy-loading 'vcpu_model' on Instance uuid 1e4dae15-f69f-4e44-959a-7480e185020b {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OZAguYvD', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:56:59 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:56:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:56:59 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:56:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:56:59 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[016b9506-37d6-45d7-abfc-01fc1a15a66e]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 03:56:59 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4e443239-07a5-4197-a41b-acfae3422439]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02] added
Apr 12 03:56:59 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f02944a1-fdf6-474c-8a10-375032206cc6]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:56:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[006549fa-091a-4dd7-af36-a54cf8e7da3e]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v OZAguYvD {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v OZAguYvD" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c2c5108d-4f3c-42f7-8d79-ba6737aa21ed]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'OZAguYvD'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:57:00 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Instance destroyed successfully.
Apr 12 03:57:00 user nova-compute[72081]: DEBUG nova.objects.instance [None req-7ea64428-ef2b-458e-91e6-05fec33cdba3 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lazy-loading 'resources' on Instance uuid 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4f6be072-e006-4d14-baaa-1ce9b08c84bc]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-7ea64428-ef2b-458e-91e6-05fec33cdba3 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:47:37Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1485022675',display_name='tempest-ServerActionsTestJSON-server-1485022675',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1485022675',id=36,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKmrI9wy1CUIwhlKuXvzk0mD7Pt5nZFvI1CsqS0do6Lk76JQOvGl5lvx9rkz7UAM9WGocGXyWSOUOuKTtYZRD2HqKUbTN4VPOoliE1yuu+s0lx7vuZu45Mj7wDb2S8Ksyw==',key_name='tempest-keypair-364276420',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:47:44Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='815b65aa5ad9442db0a087c646f96bbd',ramdisk_id='',reservation_id='r-5vy01cil',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestJSON-1898059859',owner_user_name='tempest-ServerActionsTestJSON-1898059859-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:47:44Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a7f422f1a05a4a92ba387e424850b360',uuid=8d533f5f-dbed-4878-a9b1-2ce4fae8fa13,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "address": "fa:16:3e:e2:fd:44", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2ef93926-6c", "ovs_interfaceid": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-7ea64428-ef2b-458e-91e6-05fec33cdba3 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converting VIF {"id": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "address": "fa:16:3e:e2:fd:44", "network": {"id": "9cac7f86-c0da-4b81-bdde-50662f79d265", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-1452412921-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "815b65aa5ad9442db0a087c646f96bbd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2ef93926-6c", "ovs_interfaceid": "2ef93926-6cf3-4d57-be98-ab555ac58b4c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-7ea64428-ef2b-458e-91e6-05fec33cdba3 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:e2:fd:44,bridge_name='br-int',has_traffic_filtering=True,id=2ef93926-6cf3-4d57-be98-ab555ac58b4c,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2ef93926-6c') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG os_vif [None req-7ea64428-ef2b-458e-91e6-05fec33cdba3 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:e2:fd:44,bridge_name='br-int',has_traffic_filtering=True,id=2ef93926-6cf3-4d57-be98-ab555ac58b4c,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2ef93926-6c') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap2ef93926-6c, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bd4b16b7-03fc-4ed2-a803-95132a5c4ef1]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Apr 12 03:57:00 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Apr 12 03:57:00 user nova-compute[72081]:  {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:57:00 user nova-compute[72081]: WARNING os_brick.initiator.connectors.iscsi [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Apr 12 03:57:00 user nova-compute[72081]: 
Apr 12 03:57:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:57:00 user nova-compute[72081]: INFO os_vif [None req-7ea64428-ef2b-458e-91e6-05fec33cdba3 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:e2:fd:44,bridge_name='br-int',has_traffic_filtering=True,id=2ef93926-6cf3-4d57-be98-ab555ac58b4c,network=Network(9cac7f86-c0da-4b81-bdde-50662f79d265),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2ef93926-6c')
Apr 12 03:57:00 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-7ea64428-ef2b-458e-91e6-05fec33cdba3 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Deleting instance files /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13_del
Apr 12 03:57:00 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-7ea64428-ef2b-458e-91e6-05fec33cdba3 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Deletion of /opt/stack/data/nova/instances/8d533f5f-dbed-4878-a9b1-2ce4fae8fa13_del complete
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --login" returned: 0 in 0.046s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0e7d4919-1545-4e79-a901-7a90a901e4a8]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02, portal: 172.16.0.220,3260]
Apr 12 03:57:00 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02, portal: 172.16.0.220,3260] successful.
Apr 12 03:57:00 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c695f651-ad57-475f-9de7-10bbeb3a360b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:57:00 user nova-compute[72081]: INFO nova.compute.manager [None req-7ea64428-ef2b-458e-91e6-05fec33cdba3 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Took 0.89 seconds to destroy the instance on the hypervisor.
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-7ea64428-ef2b-458e-91e6-05fec33cdba3 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[dd827e5d-9bd1-44c6-b65c-be8ec69e7915]: (4, ('tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)
Apr 12 03:57:00 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsi session list stdout=tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)
Apr 12 03:57:00 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.281s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] HCTL ('33', '-', '-', 0) found on session 49 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5a21b98a-5bdc-4d7d-a555-dec49f1c1d11]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Searching for a device in session 49 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OZAguYvD', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG nova.network.neutron [req-e3aac771-43df-431e-9a6d-77b3e718794a req-5b75bdfb-f198-489f-91bd-cb3e298f7db0 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Updated VIF entry in instance network info cache for port 1be56e0c-9ee5-4df6-bbd6-f866a4e54e90. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG nova.network.neutron [req-e3aac771-43df-431e-9a6d-77b3e718794a req-5b75bdfb-f198-489f-91bd-cb3e298f7db0 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Updating instance_info_cache with network_info: [{"id": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "address": "fa:16:3e:e9:ff:d5", "network": {"id": "99c0b9a9-e1b4-43d1-bbbb-66560eca330a", "bridge": "br-int", "label": "tempest-TestShelveInstance-2092145606-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.62", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d8ec2318ea05403db924630461ea77e5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be56e0c-9e", "ovs_interfaceid": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-e3aac771-43df-431e-9a6d-77b3e718794a req-5b75bdfb-f198-489f-91bd-cb3e298f7db0 service nova] Releasing lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:57:00 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Took 0.52 seconds to deallocate network for instance.
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7ea64428-ef2b-458e-91e6-05fec33cdba3 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7ea64428-ef2b-458e-91e6-05fec33cdba3 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-7ea64428-ef2b-458e-91e6-05fec33cdba3 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-7ea64428-ef2b-458e-91e6-05fec33cdba3 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:57:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7ea64428-ef2b-458e-91e6-05fec33cdba3 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.156s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:57:00 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-7ea64428-ef2b-458e-91e6-05fec33cdba3 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Deleted allocations for instance 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13
Apr 12 03:57:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7ea64428-ef2b-458e-91e6-05fec33cdba3 tempest-ServerActionsTestJSON-1898059859 tempest-ServerActionsTestJSON-1898059859-project-member] Lock "8d533f5f-dbed-4878-a9b1-2ce4fae8fa13" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.748s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.311s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] <== connect_volume: return (1314ms) {'type': 'block', 'scsi_wwn': '23564643731633831', 'path': '/dev/sdb'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23564643731633831', 'path': '/dev/sdb'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-12T03:53:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestShelveInstance-server-1000809041',display_name='tempest-TestShelveInstance-server-1000809041',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testshelveinstance-server-1000809041',id=42,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name='tempest-TestShelveInstance-1986276884',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:53:21Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='d8ec2318ea05403db924630461ea77e5',ramdisk_id='',reservation_id='r-3erg1hz6',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestShelveInstance-130328817',owner_user_name='tempest-TestShelveInstance-130328817-project-member'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:56:53Z,user_data=None,user_id='447437f9d71b4f8ab0da168f0fca732f',uuid=1e4dae15-f69f-4e44-959a-7480e185020b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "address": "fa:16:3e:e9:ff:d5", "network": {"id": "99c0b9a9-e1b4-43d1-bbbb-66560eca330a", "bridge": "br-int", "label": "tempest-TestShelveInstance-2092145606-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.62", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d8ec2318ea05403db924630461ea77e5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be56e0c-9e", "ovs_interfaceid": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Converting VIF {"id": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "address": "fa:16:3e:e9:ff:d5", "network": {"id": "99c0b9a9-e1b4-43d1-bbbb-66560eca330a", "bridge": "br-int", "label": "tempest-TestShelveInstance-2092145606-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.62", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d8ec2318ea05403db924630461ea77e5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be56e0c-9e", "ovs_interfaceid": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e9:ff:d5,bridge_name='br-int',has_traffic_filtering=True,id=1be56e0c-9ee5-4df6-bbd6-f866a4e54e90,network=Network(99c0b9a9-e1b4-43d1-bbbb-66560eca330a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1be56e0c-9e') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG nova.objects.instance [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lazy-loading 'pci_devices' on Instance uuid 1e4dae15-f69f-4e44-959a-7480e185020b {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG nova.compute.manager [req-414bc14d-ccb3-4dbc-8628-1e5376c1b133 req-34e4b480-ed2a-49c0-8ae4-dda530ac8631 service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Received event network-vif-deleted-2ef93926-6cf3-4d57-be98-ab555ac58b4c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:57:01 user nova-compute[72081]:   <uuid>1e4dae15-f69f-4e44-959a-7480e185020b</uuid>
Apr 12 03:57:01 user nova-compute[72081]:   <name>instance-0000002a</name>
Apr 12 03:57:01 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:57:01 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:57:01 user nova-compute[72081]:   <metadata>
Apr 12 03:57:01 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:57:01 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:57:01 user nova-compute[72081]:       <nova:name>tempest-TestShelveInstance-server-1000809041</nova:name>
Apr 12 03:57:01 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:56:59</nova:creationTime>
Apr 12 03:57:01 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:57:01 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:57:01 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:57:01 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:57:01 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:57:01 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:57:01 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:57:01 user nova-compute[72081]:       <nova:owner>
Apr 12 03:57:01 user nova-compute[72081]:         <nova:user uuid="447437f9d71b4f8ab0da168f0fca732f">tempest-TestShelveInstance-130328817-project-member</nova:user>
Apr 12 03:57:01 user nova-compute[72081]:         <nova:project uuid="d8ec2318ea05403db924630461ea77e5">tempest-TestShelveInstance-130328817</nova:project>
Apr 12 03:57:01 user nova-compute[72081]:       </nova:owner>
Apr 12 03:57:01 user nova-compute[72081]:       <nova:ports>
Apr 12 03:57:01 user nova-compute[72081]:         <nova:port uuid="1be56e0c-9ee5-4df6-bbd6-f866a4e54e90">
Apr 12 03:57:01 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Apr 12 03:57:01 user nova-compute[72081]:         </nova:port>
Apr 12 03:57:01 user nova-compute[72081]:       </nova:ports>
Apr 12 03:57:01 user nova-compute[72081]:     </nova:instance>
Apr 12 03:57:01 user nova-compute[72081]:   </metadata>
Apr 12 03:57:01 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:57:01 user nova-compute[72081]:     <system>
Apr 12 03:57:01 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:57:01 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:57:01 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:57:01 user nova-compute[72081]:       <entry name="serial">1e4dae15-f69f-4e44-959a-7480e185020b</entry>
Apr 12 03:57:01 user nova-compute[72081]:       <entry name="uuid">1e4dae15-f69f-4e44-959a-7480e185020b</entry>
Apr 12 03:57:01 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:57:01 user nova-compute[72081]:     </system>
Apr 12 03:57:01 user nova-compute[72081]:   </sysinfo>
Apr 12 03:57:01 user nova-compute[72081]:   <os>
Apr 12 03:57:01 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:57:01 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:57:01 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:57:01 user nova-compute[72081]:   </os>
Apr 12 03:57:01 user nova-compute[72081]:   <features>
Apr 12 03:57:01 user nova-compute[72081]:     <acpi/>
Apr 12 03:57:01 user nova-compute[72081]:     <apic/>
Apr 12 03:57:01 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:57:01 user nova-compute[72081]:   </features>
Apr 12 03:57:01 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:57:01 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:57:01 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:57:01 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:57:01 user nova-compute[72081]:   </clock>
Apr 12 03:57:01 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:57:01 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:57:01 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:57:01 user nova-compute[72081]:   </cpu>
Apr 12 03:57:01 user nova-compute[72081]:   <devices>
Apr 12 03:57:01 user nova-compute[72081]:     <disk type="block" device="disk">
Apr 12 03:57:01 user nova-compute[72081]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 03:57:01 user nova-compute[72081]:       <source dev="/dev/sdb"/>
Apr 12 03:57:01 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:57:01 user nova-compute[72081]:       <serial>d6a07e3a-be2a-4191-b1b7-09667064fb02</serial>
Apr 12 03:57:01 user nova-compute[72081]:     </disk>
Apr 12 03:57:01 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:57:01 user nova-compute[72081]:       <mac address="fa:16:3e:e9:ff:d5"/>
Apr 12 03:57:01 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:57:01 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:57:01 user nova-compute[72081]:       <target dev="tap1be56e0c-9e"/>
Apr 12 03:57:01 user nova-compute[72081]:     </interface>
Apr 12 03:57:01 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:57:01 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/1e4dae15-f69f-4e44-959a-7480e185020b/console.log" append="off"/>
Apr 12 03:57:01 user nova-compute[72081]:     </serial>
Apr 12 03:57:01 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:57:01 user nova-compute[72081]:     <video>
Apr 12 03:57:01 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:57:01 user nova-compute[72081]:     </video>
Apr 12 03:57:01 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:57:01 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:57:01 user nova-compute[72081]:     </rng>
Apr 12 03:57:01 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:57:01 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:57:01 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:57:01 user nova-compute[72081]:     </memballoon>
Apr 12 03:57:01 user nova-compute[72081]:   </devices>
Apr 12 03:57:01 user nova-compute[72081]: </domain>
Apr 12 03:57:01 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-12T03:53:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestShelveInstance-server-1000809041',display_name='tempest-TestShelveInstance-server-1000809041',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testshelveinstance-server-1000809041',id=42,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name='tempest-TestShelveInstance-1986276884',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:53:21Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='d8ec2318ea05403db924630461ea77e5',ramdisk_id='',reservation_id='r-3erg1hz6',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestShelveInstance-130328817',owner_user_name='tempest-TestShelveInstance-130328817-project-member'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:56:53Z,user_data=None,user_id='447437f9d71b4f8ab0da168f0fca732f',uuid=1e4dae15-f69f-4e44-959a-7480e185020b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "address": "fa:16:3e:e9:ff:d5", "network": {"id": "99c0b9a9-e1b4-43d1-bbbb-66560eca330a", "bridge": "br-int", "label": "tempest-TestShelveInstance-2092145606-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.62", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d8ec2318ea05403db924630461ea77e5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be56e0c-9e", "ovs_interfaceid": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Converting VIF {"id": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "address": "fa:16:3e:e9:ff:d5", "network": {"id": "99c0b9a9-e1b4-43d1-bbbb-66560eca330a", "bridge": "br-int", "label": "tempest-TestShelveInstance-2092145606-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.62", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d8ec2318ea05403db924630461ea77e5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be56e0c-9e", "ovs_interfaceid": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e9:ff:d5,bridge_name='br-int',has_traffic_filtering=True,id=1be56e0c-9ee5-4df6-bbd6-f866a4e54e90,network=Network(99c0b9a9-e1b4-43d1-bbbb-66560eca330a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1be56e0c-9e') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG os_vif [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:e9:ff:d5,bridge_name='br-int',has_traffic_filtering=True,id=1be56e0c-9ee5-4df6-bbd6-f866a4e54e90,network=Network(99c0b9a9-e1b4-43d1-bbbb-66560eca330a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1be56e0c-9e') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap1be56e0c-9e, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap1be56e0c-9e, col_values=(('external_ids', {'iface-id': '1be56e0c-9ee5-4df6-bbd6-f866a4e54e90', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:e9:ff:d5', 'vm-uuid': '1e4dae15-f69f-4e44-959a-7480e185020b'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:01 user nova-compute[72081]: INFO os_vif [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:e9:ff:d5,bridge_name='br-int',has_traffic_filtering=True,id=1be56e0c-9ee5-4df6-bbd6-f866a4e54e90,network=Network(99c0b9a9-e1b4-43d1-bbbb-66560eca330a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1be56e0c-9e')
Apr 12 03:57:01 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] No VIF found with MAC fa:16:3e:e9:ff:d5, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG nova.compute.manager [req-58e11c15-275f-45e5-a3c4-4738291ef383 req-5b830383-aa0a-46bc-bb71-630d21e8be24 service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Received event network-vif-plugged-2ef93926-6cf3-4d57-be98-ab555ac58b4c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-58e11c15-275f-45e5-a3c4-4738291ef383 req-5b830383-aa0a-46bc-bb71-630d21e8be24 service nova] Acquiring lock "8d533f5f-dbed-4878-a9b1-2ce4fae8fa13-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-58e11c15-275f-45e5-a3c4-4738291ef383 req-5b830383-aa0a-46bc-bb71-630d21e8be24 service nova] Lock "8d533f5f-dbed-4878-a9b1-2ce4fae8fa13-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-58e11c15-275f-45e5-a3c4-4738291ef383 req-5b830383-aa0a-46bc-bb71-630d21e8be24 service nova] Lock "8d533f5f-dbed-4878-a9b1-2ce4fae8fa13-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:57:01 user nova-compute[72081]: DEBUG nova.compute.manager [req-58e11c15-275f-45e5-a3c4-4738291ef383 req-5b830383-aa0a-46bc-bb71-630d21e8be24 service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] No waiting events found dispatching network-vif-plugged-2ef93926-6cf3-4d57-be98-ab555ac58b4c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:57:01 user nova-compute[72081]: WARNING nova.compute.manager [req-58e11c15-275f-45e5-a3c4-4738291ef383 req-5b830383-aa0a-46bc-bb71-630d21e8be24 service nova] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Received unexpected event network-vif-plugged-2ef93926-6cf3-4d57-be98-ab555ac58b4c for instance with vm_state deleted and task_state None.
Apr 12 03:57:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:02 user nova-compute[72081]: DEBUG nova.compute.manager [req-01d595d5-bba7-4af6-95e9-314eabbe3cd9 req-a9e4524a-5fd4-47a1-be81-6fd70fcfcaa0 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Received event network-vif-plugged-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:57:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-01d595d5-bba7-4af6-95e9-314eabbe3cd9 req-a9e4524a-5fd4-47a1-be81-6fd70fcfcaa0 service nova] Acquiring lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:57:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-01d595d5-bba7-4af6-95e9-314eabbe3cd9 req-a9e4524a-5fd4-47a1-be81-6fd70fcfcaa0 service nova] Lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:57:02 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-01d595d5-bba7-4af6-95e9-314eabbe3cd9 req-a9e4524a-5fd4-47a1-be81-6fd70fcfcaa0 service nova] Lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:57:02 user nova-compute[72081]: DEBUG nova.compute.manager [req-01d595d5-bba7-4af6-95e9-314eabbe3cd9 req-a9e4524a-5fd4-47a1-be81-6fd70fcfcaa0 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] No waiting events found dispatching network-vif-plugged-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:57:02 user nova-compute[72081]: WARNING nova.compute.manager [req-01d595d5-bba7-4af6-95e9-314eabbe3cd9 req-a9e4524a-5fd4-47a1-be81-6fd70fcfcaa0 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Received unexpected event network-vif-plugged-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 for instance with vm_state shelved_offloaded and task_state spawning.
Apr 12 03:57:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:04 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271824.6322613, 1e4dae15-f69f-4e44-959a-7480e185020b => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:57:04 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] VM Resumed (Lifecycle Event)
Apr 12 03:57:04 user nova-compute[72081]: DEBUG nova.compute.manager [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:57:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:57:04 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Instance spawned successfully.
Apr 12 03:57:04 user nova-compute[72081]: DEBUG nova.compute.manager [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:57:04 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:57:04 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: shelved_offloaded, current task_state: spawning, current DB power_state: 4, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:57:04 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:57:04 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271824.6357205, 1e4dae15-f69f-4e44-959a-7480e185020b => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:57:04 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] VM Started (Lifecycle Event)
Apr 12 03:57:04 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:57:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a7af2195-8e4b-4dfd-a3fa-646955cad4c7 tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "1e4dae15-f69f-4e44-959a-7480e185020b" "released" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: held 11.371s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:57:04 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:57:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-a866a60c-5de8-49f6-96f7-14ab439787de req-09a17ed2-87f4-4399-bd79-75b0cf3ef5c6 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Received event network-vif-plugged-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:57:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a866a60c-5de8-49f6-96f7-14ab439787de req-09a17ed2-87f4-4399-bd79-75b0cf3ef5c6 service nova] Acquiring lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:57:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a866a60c-5de8-49f6-96f7-14ab439787de req-09a17ed2-87f4-4399-bd79-75b0cf3ef5c6 service nova] Lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:57:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a866a60c-5de8-49f6-96f7-14ab439787de req-09a17ed2-87f4-4399-bd79-75b0cf3ef5c6 service nova] Lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:57:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-a866a60c-5de8-49f6-96f7-14ab439787de req-09a17ed2-87f4-4399-bd79-75b0cf3ef5c6 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] No waiting events found dispatching network-vif-plugged-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:57:04 user nova-compute[72081]: WARNING nova.compute.manager [req-a866a60c-5de8-49f6-96f7-14ab439787de req-09a17ed2-87f4-4399-bd79-75b0cf3ef5c6 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Received unexpected event network-vif-plugged-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 for instance with vm_state active and task_state None.
Apr 12 03:57:06 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:15 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681271820.051419, 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:57:15 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] VM Stopped (Lifecycle Event)
Apr 12 03:57:15 user nova-compute[72081]: DEBUG nova.compute.manager [None req-d8bfe67b-2801-4cad-8fe5-19097fadcdd9 None None] [instance: 8d533f5f-dbed-4878-a9b1-2ce4fae8fa13] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:57:16 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:57:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:19 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_power_states {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:57:19 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Triggering sync for uuid faa9190f-a85c-4107-9c7c-3de959b5d4d0 {{(pid=72081) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 12 03:57:19 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Triggering sync for uuid 1e4dae15-f69f-4e44-959a-7480e185020b {{(pid=72081) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 12 03:57:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "faa9190f-a85c-4107-9c7c-3de959b5d4d0" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:57:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "faa9190f-a85c-4107-9c7c-3de959b5d4d0" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:57:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "1e4dae15-f69f-4e44-959a-7480e185020b" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:57:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "1e4dae15-f69f-4e44-959a-7480e185020b" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:57:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "faa9190f-a85c-4107-9c7c-3de959b5d4d0" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.030s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:57:20 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "1e4dae15-f69f-4e44-959a-7480e185020b" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.031s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:57:20 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:57:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:21 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:57:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:24 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:57:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:57:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:57:24 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:57:24 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:57:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/faa9190f-a85c-4107-9c7c-3de959b5d4d0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:57:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/faa9190f-a85c-4107-9c7c-3de959b5d4d0/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:57:24 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/faa9190f-a85c-4107-9c7c-3de959b5d4d0/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:57:25 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/faa9190f-a85c-4107-9c7c-3de959b5d4d0/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:57:25 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:57:25 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:57:25 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:57:25 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8597MB free_disk=26.040294647216797GB free_vcpus=10 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:57:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:57:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:57:25 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance faa9190f-a85c-4107-9c7c-3de959b5d4d0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:57:25 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 1e4dae15-f69f-4e44-959a-7480e185020b actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:57:25 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 2 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:57:25 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=768MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=2 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:57:25 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:57:25 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:57:25 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:57:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.227s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:57:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:27 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:57:27 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:57:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:28 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:57:28 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:57:30 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:57:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:31 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:57:31 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:57:31 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Rebuilding the list of instances to heal {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 12 03:57:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-faa9190f-a85c-4107-9c7c-3de959b5d4d0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:57:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-faa9190f-a85c-4107-9c7c-3de959b5d4d0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:57:31 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 03:57:31 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lazy-loading 'info_cache' on Instance uuid faa9190f-a85c-4107-9c7c-3de959b5d4d0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:57:32 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Updating instance_info_cache with network_info: [{"id": "8531b3ef-8643-4221-8728-14adc021e84f", "address": "fa:16:3e:0d:2b:d9", "network": {"id": "123991b1-68ee-4b1b-8ce9-35b830ec5822", "bridge": "br-int", "label": "tempest-VolumesActionsTest-820420474-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "b4b1260a71aa436081e636ca7b996ead", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8531b3ef-86", "ovs_interfaceid": "8531b3ef-8643-4221-8728-14adc021e84f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:57:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-faa9190f-a85c-4107-9c7c-3de959b5d4d0" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:57:32 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 03:57:32 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:57:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:57:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:57:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 03:57:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 03:57:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 03:57:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:57:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c4af9ac2-ed58-4a3e-b530-0270fd21cd42 tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Acquiring lock "faa9190f-a85c-4107-9c7c-3de959b5d4d0" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:57:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c4af9ac2-ed58-4a3e-b530-0270fd21cd42 tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Lock "faa9190f-a85c-4107-9c7c-3de959b5d4d0" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:57:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c4af9ac2-ed58-4a3e-b530-0270fd21cd42 tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Acquiring lock "faa9190f-a85c-4107-9c7c-3de959b5d4d0-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:57:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c4af9ac2-ed58-4a3e-b530-0270fd21cd42 tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Lock "faa9190f-a85c-4107-9c7c-3de959b5d4d0-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:57:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c4af9ac2-ed58-4a3e-b530-0270fd21cd42 tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Lock "faa9190f-a85c-4107-9c7c-3de959b5d4d0-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:57:56 user nova-compute[72081]: INFO nova.compute.manager [None req-c4af9ac2-ed58-4a3e-b530-0270fd21cd42 tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Terminating instance
Apr 12 03:57:56 user nova-compute[72081]: DEBUG nova.compute.manager [None req-c4af9ac2-ed58-4a3e-b530-0270fd21cd42 tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:57:56 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:57 user nova-compute[72081]: DEBUG nova.compute.manager [req-0d32de62-0eb8-48b9-af35-cfd59b3b6c87 req-1661a055-1a6b-40c3-bb81-a4121d44624d service nova] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Received event network-vif-unplugged-8531b3ef-8643-4221-8728-14adc021e84f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:57:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0d32de62-0eb8-48b9-af35-cfd59b3b6c87 req-1661a055-1a6b-40c3-bb81-a4121d44624d service nova] Acquiring lock "faa9190f-a85c-4107-9c7c-3de959b5d4d0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:57:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0d32de62-0eb8-48b9-af35-cfd59b3b6c87 req-1661a055-1a6b-40c3-bb81-a4121d44624d service nova] Lock "faa9190f-a85c-4107-9c7c-3de959b5d4d0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:57:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0d32de62-0eb8-48b9-af35-cfd59b3b6c87 req-1661a055-1a6b-40c3-bb81-a4121d44624d service nova] Lock "faa9190f-a85c-4107-9c7c-3de959b5d4d0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:57:57 user nova-compute[72081]: DEBUG nova.compute.manager [req-0d32de62-0eb8-48b9-af35-cfd59b3b6c87 req-1661a055-1a6b-40c3-bb81-a4121d44624d service nova] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] No waiting events found dispatching network-vif-unplugged-8531b3ef-8643-4221-8728-14adc021e84f {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:57:57 user nova-compute[72081]: DEBUG nova.compute.manager [req-0d32de62-0eb8-48b9-af35-cfd59b3b6c87 req-1661a055-1a6b-40c3-bb81-a4121d44624d service nova] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Received event network-vif-unplugged-8531b3ef-8643-4221-8728-14adc021e84f for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:57:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:57 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Instance destroyed successfully.
Apr 12 03:57:57 user nova-compute[72081]: DEBUG nova.objects.instance [None req-c4af9ac2-ed58-4a3e-b530-0270fd21cd42 tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Lazy-loading 'resources' on Instance uuid faa9190f-a85c-4107-9c7c-3de959b5d4d0 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:57:57 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-c4af9ac2-ed58-4a3e-b530-0270fd21cd42 tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:56:01Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-480847617',display_name='tempest-VolumesActionsTest-instance-480847617',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-480847617',id=43,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:56:07Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='b4b1260a71aa436081e636ca7b996ead',ramdisk_id='',reservation_id='r-y7vjjd74',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesActionsTest-437031612',owner_user_name='tempest-VolumesActionsTest-437031612-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:56:08Z,user_data=None,user_id='79bd18d6965c4f85bbbc121864fe5782',uuid=faa9190f-a85c-4107-9c7c-3de959b5d4d0,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "8531b3ef-8643-4221-8728-14adc021e84f", "address": "fa:16:3e:0d:2b:d9", "network": {"id": "123991b1-68ee-4b1b-8ce9-35b830ec5822", "bridge": "br-int", "label": "tempest-VolumesActionsTest-820420474-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "b4b1260a71aa436081e636ca7b996ead", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8531b3ef-86", "ovs_interfaceid": "8531b3ef-8643-4221-8728-14adc021e84f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:57:57 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-c4af9ac2-ed58-4a3e-b530-0270fd21cd42 tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Converting VIF {"id": "8531b3ef-8643-4221-8728-14adc021e84f", "address": "fa:16:3e:0d:2b:d9", "network": {"id": "123991b1-68ee-4b1b-8ce9-35b830ec5822", "bridge": "br-int", "label": "tempest-VolumesActionsTest-820420474-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "b4b1260a71aa436081e636ca7b996ead", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8531b3ef-86", "ovs_interfaceid": "8531b3ef-8643-4221-8728-14adc021e84f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:57:57 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-c4af9ac2-ed58-4a3e-b530-0270fd21cd42 tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:0d:2b:d9,bridge_name='br-int',has_traffic_filtering=True,id=8531b3ef-8643-4221-8728-14adc021e84f,network=Network(123991b1-68ee-4b1b-8ce9-35b830ec5822),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8531b3ef-86') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:57:57 user nova-compute[72081]: DEBUG os_vif [None req-c4af9ac2-ed58-4a3e-b530-0270fd21cd42 tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:0d:2b:d9,bridge_name='br-int',has_traffic_filtering=True,id=8531b3ef-8643-4221-8728-14adc021e84f,network=Network(123991b1-68ee-4b1b-8ce9-35b830ec5822),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8531b3ef-86') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:57:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap8531b3ef-86, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:57:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:57:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:57:57 user nova-compute[72081]: INFO os_vif [None req-c4af9ac2-ed58-4a3e-b530-0270fd21cd42 tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:0d:2b:d9,bridge_name='br-int',has_traffic_filtering=True,id=8531b3ef-8643-4221-8728-14adc021e84f,network=Network(123991b1-68ee-4b1b-8ce9-35b830ec5822),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8531b3ef-86')
Apr 12 03:57:57 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-c4af9ac2-ed58-4a3e-b530-0270fd21cd42 tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Deleting instance files /opt/stack/data/nova/instances/faa9190f-a85c-4107-9c7c-3de959b5d4d0_del
Apr 12 03:57:57 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-c4af9ac2-ed58-4a3e-b530-0270fd21cd42 tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Deletion of /opt/stack/data/nova/instances/faa9190f-a85c-4107-9c7c-3de959b5d4d0_del complete
Apr 12 03:57:57 user nova-compute[72081]: INFO nova.compute.manager [None req-c4af9ac2-ed58-4a3e-b530-0270fd21cd42 tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Took 0.68 seconds to destroy the instance on the hypervisor.
Apr 12 03:57:57 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-c4af9ac2-ed58-4a3e-b530-0270fd21cd42 tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:57:57 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:57:57 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:57:58 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:57:58 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Took 0.78 seconds to deallocate network for instance.
Apr 12 03:57:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-7670b7ee-a1bf-4302-aacf-8de063b6203a req-5e2143cc-87e3-42be-9fc0-fb349e6e3a80 service nova] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Received event network-vif-deleted-8531b3ef-8643-4221-8728-14adc021e84f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:57:58 user nova-compute[72081]: INFO nova.compute.manager [req-7670b7ee-a1bf-4302-aacf-8de063b6203a req-5e2143cc-87e3-42be-9fc0-fb349e6e3a80 service nova] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Neutron deleted interface 8531b3ef-8643-4221-8728-14adc021e84f; detaching it from the instance and deleting it from the info cache
Apr 12 03:57:58 user nova-compute[72081]: DEBUG nova.network.neutron [req-7670b7ee-a1bf-4302-aacf-8de063b6203a req-5e2143cc-87e3-42be-9fc0-fb349e6e3a80 service nova] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:57:58 user nova-compute[72081]: DEBUG nova.compute.manager [req-7670b7ee-a1bf-4302-aacf-8de063b6203a req-5e2143cc-87e3-42be-9fc0-fb349e6e3a80 service nova] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Detach interface failed, port_id=8531b3ef-8643-4221-8728-14adc021e84f, reason: Instance faa9190f-a85c-4107-9c7c-3de959b5d4d0 could not be found. {{(pid=72081) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 12 03:57:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c4af9ac2-ed58-4a3e-b530-0270fd21cd42 tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:57:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c4af9ac2-ed58-4a3e-b530-0270fd21cd42 tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:57:58 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-c4af9ac2-ed58-4a3e-b530-0270fd21cd42 tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:57:58 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-c4af9ac2-ed58-4a3e-b530-0270fd21cd42 tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:57:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c4af9ac2-ed58-4a3e-b530-0270fd21cd42 tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.145s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:57:58 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-c4af9ac2-ed58-4a3e-b530-0270fd21cd42 tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Deleted allocations for instance faa9190f-a85c-4107-9c7c-3de959b5d4d0
Apr 12 03:57:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c4af9ac2-ed58-4a3e-b530-0270fd21cd42 tempest-VolumesActionsTest-437031612 tempest-VolumesActionsTest-437031612-project-member] Lock "faa9190f-a85c-4107-9c7c-3de959b5d4d0" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.792s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:57:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-35ed6276-0e46-4651-97bb-b83ed88121f8 req-affc12f2-e54c-4bab-8053-58be5099a5ce service nova] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Received event network-vif-plugged-8531b3ef-8643-4221-8728-14adc021e84f {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:57:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-35ed6276-0e46-4651-97bb-b83ed88121f8 req-affc12f2-e54c-4bab-8053-58be5099a5ce service nova] Acquiring lock "faa9190f-a85c-4107-9c7c-3de959b5d4d0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:57:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-35ed6276-0e46-4651-97bb-b83ed88121f8 req-affc12f2-e54c-4bab-8053-58be5099a5ce service nova] Lock "faa9190f-a85c-4107-9c7c-3de959b5d4d0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:57:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-35ed6276-0e46-4651-97bb-b83ed88121f8 req-affc12f2-e54c-4bab-8053-58be5099a5ce service nova] Lock "faa9190f-a85c-4107-9c7c-3de959b5d4d0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:57:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-35ed6276-0e46-4651-97bb-b83ed88121f8 req-affc12f2-e54c-4bab-8053-58be5099a5ce service nova] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] No waiting events found dispatching network-vif-plugged-8531b3ef-8643-4221-8728-14adc021e84f {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:57:59 user nova-compute[72081]: WARNING nova.compute.manager [req-35ed6276-0e46-4651-97bb-b83ed88121f8 req-affc12f2-e54c-4bab-8053-58be5099a5ce service nova] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Received unexpected event network-vif-plugged-8531b3ef-8643-4221-8728-14adc021e84f for instance with vm_state deleted and task_state None.
Apr 12 03:58:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "de40e137-84e3-4d85-895b-1f6da2673457" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:58:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "de40e137-84e3-4d85-895b-1f6da2673457" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:58:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:58:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:58:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:58:07 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:58:07 user nova-compute[72081]: INFO nova.compute.claims [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Claim successful on node user
Apr 12 03:58:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:07 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:58:07 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:58:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.205s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:58:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:58:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:58:07 user nova-compute[72081]: DEBUG nova.network.neutron [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:58:07 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:58:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:58:07 user nova-compute[72081]: DEBUG nova.policy [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '44a0206f1323440db4f8411a3c3c088d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a6a8a4f5d59e479f9ca7b73a9e270c4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:58:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:58:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:58:07 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Creating image(s)
Apr 12 03:58:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "/opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:58:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "/opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:58:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "/opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:58:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:58:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:58:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:58:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:58:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "6cd92d70-a2af-4b2a-97c3-2981c157278d" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "6cd92d70-a2af-4b2a-97c3-2981c157278d" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.143s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk 1073741824" returned: 0 in 0.052s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.207s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:58:08 user nova-compute[72081]: INFO nova.compute.claims [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Claim successful on node user
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.133s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Checking if we can resize image /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG nova.network.neutron [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Successfully created port: 16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Cannot resize image /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG nova.objects.instance [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'migration_context' on Instance uuid de40e137-84e3-4d85-895b-1f6da2673457 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Ensure instance console log exists: /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.330s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:58:08 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:58:08 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:58:08 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Creating image(s)
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "/opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "/opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "/opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG nova.policy [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '44a0206f1323440db4f8411a3c3c088d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a6a8a4f5d59e479f9ca7b73a9e270c4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.132s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.148s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:58:08 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk 1073741824" returned: 0 in 0.045s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.200s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.155s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Checking if we can resize image /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.network.neutron [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Successfully updated port: 16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquired lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.network.neutron [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-52c99640-e322-4e9d-8eb9-e7b35aaa27f7 req-d195a0d6-2001-4b28-be30-1e43aa445963 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received event network-changed-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-52c99640-e322-4e9d-8eb9-e7b35aaa27f7 req-d195a0d6-2001-4b28-be30-1e43aa445963 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Refreshing instance network info cache due to event network-changed-16a3f234-3d0d-446c-b6a0-0a240e35aa6c. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-52c99640-e322-4e9d-8eb9-e7b35aaa27f7 req-d195a0d6-2001-4b28-be30-1e43aa445963 service nova] Acquiring lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Cannot resize image /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'migration_context' on Instance uuid 6cd92d70-a2af-4b2a-97c3-2981c157278d {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Ensure instance console log exists: /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.network.neutron [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Successfully created port: ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.network.neutron [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Updating instance_info_cache with network_info: [{"id": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "address": "fa:16:3e:c4:e2:d7", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap16a3f234-3d", "ovs_interfaceid": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Releasing lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.compute.manager [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Instance network_info: |[{"id": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "address": "fa:16:3e:c4:e2:d7", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap16a3f234-3d", "ovs_interfaceid": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-52c99640-e322-4e9d-8eb9-e7b35aaa27f7 req-d195a0d6-2001-4b28-be30-1e43aa445963 service nova] Acquired lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.network.neutron [req-52c99640-e322-4e9d-8eb9-e7b35aaa27f7 req-d195a0d6-2001-4b28-be30-1e43aa445963 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Refreshing network info cache for port 16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Start _get_guest_xml network_info=[{"id": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "address": "fa:16:3e:c4:e2:d7", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap16a3f234-3d", "ovs_interfaceid": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:58:09 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:58:09 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:58:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-1232389953',display_name='tempest-ServerRescueNegativeTestJSON-server-1232389953',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-1232389953',id=44,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a6a8a4f5d59e479f9ca7b73a9e270c4b',ramdisk_id='',reservation_id='r-tsy8s5du',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-529226261',owner_user_name='tempest-ServerRescueNegativeTestJSON-529226261-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:58:08Z,user_data=None,user_id='44a0206f1323440db4f8411a3c3c088d',uuid=de40e137-84e3-4d85-895b-1f6da2673457,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "address": "fa:16:3e:c4:e2:d7", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap16a3f234-3d", "ovs_interfaceid": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Converting VIF {"id": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "address": "fa:16:3e:c4:e2:d7", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap16a3f234-3d", "ovs_interfaceid": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:c4:e2:d7,bridge_name='br-int',has_traffic_filtering=True,id=16a3f234-3d0d-446c-b6a0-0a240e35aa6c,network=Network(311bda05-075c-4bb3-9469-b5117dd33353),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap16a3f234-3d') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:58:09 user nova-compute[72081]: DEBUG nova.objects.instance [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'pci_devices' on Instance uuid de40e137-84e3-4d85-895b-1f6da2673457 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:58:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:58:10 user nova-compute[72081]:   <uuid>de40e137-84e3-4d85-895b-1f6da2673457</uuid>
Apr 12 03:58:10 user nova-compute[72081]:   <name>instance-0000002c</name>
Apr 12 03:58:10 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:58:10 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:58:10 user nova-compute[72081]:   <metadata>
Apr 12 03:58:10 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:58:10 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:58:10 user nova-compute[72081]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-1232389953</nova:name>
Apr 12 03:58:10 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:58:09</nova:creationTime>
Apr 12 03:58:10 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:58:10 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:58:10 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:58:10 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:58:10 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:58:10 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:58:10 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:58:10 user nova-compute[72081]:       <nova:owner>
Apr 12 03:58:10 user nova-compute[72081]:         <nova:user uuid="44a0206f1323440db4f8411a3c3c088d">tempest-ServerRescueNegativeTestJSON-529226261-project-member</nova:user>
Apr 12 03:58:10 user nova-compute[72081]:         <nova:project uuid="a6a8a4f5d59e479f9ca7b73a9e270c4b">tempest-ServerRescueNegativeTestJSON-529226261</nova:project>
Apr 12 03:58:10 user nova-compute[72081]:       </nova:owner>
Apr 12 03:58:10 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:58:10 user nova-compute[72081]:       <nova:ports>
Apr 12 03:58:10 user nova-compute[72081]:         <nova:port uuid="16a3f234-3d0d-446c-b6a0-0a240e35aa6c">
Apr 12 03:58:10 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Apr 12 03:58:10 user nova-compute[72081]:         </nova:port>
Apr 12 03:58:10 user nova-compute[72081]:       </nova:ports>
Apr 12 03:58:10 user nova-compute[72081]:     </nova:instance>
Apr 12 03:58:10 user nova-compute[72081]:   </metadata>
Apr 12 03:58:10 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:58:10 user nova-compute[72081]:     <system>
Apr 12 03:58:10 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:58:10 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:58:10 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:58:10 user nova-compute[72081]:       <entry name="serial">de40e137-84e3-4d85-895b-1f6da2673457</entry>
Apr 12 03:58:10 user nova-compute[72081]:       <entry name="uuid">de40e137-84e3-4d85-895b-1f6da2673457</entry>
Apr 12 03:58:10 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:58:10 user nova-compute[72081]:     </system>
Apr 12 03:58:10 user nova-compute[72081]:   </sysinfo>
Apr 12 03:58:10 user nova-compute[72081]:   <os>
Apr 12 03:58:10 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:58:10 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:58:10 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:58:10 user nova-compute[72081]:   </os>
Apr 12 03:58:10 user nova-compute[72081]:   <features>
Apr 12 03:58:10 user nova-compute[72081]:     <acpi/>
Apr 12 03:58:10 user nova-compute[72081]:     <apic/>
Apr 12 03:58:10 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:58:10 user nova-compute[72081]:   </features>
Apr 12 03:58:10 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:58:10 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:58:10 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:58:10 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:58:10 user nova-compute[72081]:   </clock>
Apr 12 03:58:10 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:58:10 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:58:10 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:58:10 user nova-compute[72081]:   </cpu>
Apr 12 03:58:10 user nova-compute[72081]:   <devices>
Apr 12 03:58:10 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:58:10 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:58:10 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk"/>
Apr 12 03:58:10 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:58:10 user nova-compute[72081]:     </disk>
Apr 12 03:58:10 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:58:10 user nova-compute[72081]:       <mac address="fa:16:3e:c4:e2:d7"/>
Apr 12 03:58:10 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:58:10 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:58:10 user nova-compute[72081]:       <target dev="tap16a3f234-3d"/>
Apr 12 03:58:10 user nova-compute[72081]:     </interface>
Apr 12 03:58:10 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:58:10 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/console.log" append="off"/>
Apr 12 03:58:10 user nova-compute[72081]:     </serial>
Apr 12 03:58:10 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:58:10 user nova-compute[72081]:     <video>
Apr 12 03:58:10 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:58:10 user nova-compute[72081]:     </video>
Apr 12 03:58:10 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:58:10 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:58:10 user nova-compute[72081]:     </rng>
Apr 12 03:58:10 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:58:10 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:58:10 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:58:10 user nova-compute[72081]:     </memballoon>
Apr 12 03:58:10 user nova-compute[72081]:   </devices>
Apr 12 03:58:10 user nova-compute[72081]: </domain>
Apr 12 03:58:10 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:58:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:58:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-1232389953',display_name='tempest-ServerRescueNegativeTestJSON-server-1232389953',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-1232389953',id=44,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a6a8a4f5d59e479f9ca7b73a9e270c4b',ramdisk_id='',reservation_id='r-tsy8s5du',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-529226261',owner_user_name='tempest-ServerRescueNegativeTestJSON-529226261-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:58:08Z,user_data=None,user_id='44a0206f1323440db4f8411a3c3c088d',uuid=de40e137-84e3-4d85-895b-1f6da2673457,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "address": "fa:16:3e:c4:e2:d7", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap16a3f234-3d", "ovs_interfaceid": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:58:10 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Converting VIF {"id": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "address": "fa:16:3e:c4:e2:d7", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap16a3f234-3d", "ovs_interfaceid": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:58:10 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:c4:e2:d7,bridge_name='br-int',has_traffic_filtering=True,id=16a3f234-3d0d-446c-b6a0-0a240e35aa6c,network=Network(311bda05-075c-4bb3-9469-b5117dd33353),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap16a3f234-3d') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:58:10 user nova-compute[72081]: DEBUG os_vif [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:c4:e2:d7,bridge_name='br-int',has_traffic_filtering=True,id=16a3f234-3d0d-446c-b6a0-0a240e35aa6c,network=Network(311bda05-075c-4bb3-9469-b5117dd33353),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap16a3f234-3d') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:58:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:58:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:58:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap16a3f234-3d, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:58:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap16a3f234-3d, col_values=(('external_ids', {'iface-id': '16a3f234-3d0d-446c-b6a0-0a240e35aa6c', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:c4:e2:d7', 'vm-uuid': 'de40e137-84e3-4d85-895b-1f6da2673457'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:58:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:58:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:10 user nova-compute[72081]: INFO os_vif [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:c4:e2:d7,bridge_name='br-int',has_traffic_filtering=True,id=16a3f234-3d0d-446c-b6a0-0a240e35aa6c,network=Network(311bda05-075c-4bb3-9469-b5117dd33353),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap16a3f234-3d')
Apr 12 03:58:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:58:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] No VIF found with MAC fa:16:3e:c4:e2:d7, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:58:10 user nova-compute[72081]: DEBUG nova.network.neutron [req-52c99640-e322-4e9d-8eb9-e7b35aaa27f7 req-d195a0d6-2001-4b28-be30-1e43aa445963 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Updated VIF entry in instance network info cache for port 16a3f234-3d0d-446c-b6a0-0a240e35aa6c. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:58:10 user nova-compute[72081]: DEBUG nova.network.neutron [req-52c99640-e322-4e9d-8eb9-e7b35aaa27f7 req-d195a0d6-2001-4b28-be30-1e43aa445963 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Updating instance_info_cache with network_info: [{"id": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "address": "fa:16:3e:c4:e2:d7", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap16a3f234-3d", "ovs_interfaceid": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:58:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-52c99640-e322-4e9d-8eb9-e7b35aaa27f7 req-d195a0d6-2001-4b28-be30-1e43aa445963 service nova] Releasing lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Successfully updated port: ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "refresh_cache-6cd92d70-a2af-4b2a-97c3-2981c157278d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquired lock "refresh_cache-6cd92d70-a2af-4b2a-97c3-2981c157278d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.compute.manager [req-1f4707e7-0b29-450d-bd3a-18eb5c6e59f6 req-fbacfbd0-0b06-40eb-b50a-20b5ed2a246c service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Received event network-changed-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.compute.manager [req-1f4707e7-0b29-450d-bd3a-18eb5c6e59f6 req-fbacfbd0-0b06-40eb-b50a-20b5ed2a246c service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Refreshing instance network info cache due to event network-changed-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1f4707e7-0b29-450d-bd3a-18eb5c6e59f6 req-fbacfbd0-0b06-40eb-b50a-20b5ed2a246c service nova] Acquiring lock "refresh_cache-6cd92d70-a2af-4b2a-97c3-2981c157278d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Updating instance_info_cache with network_info: [{"id": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "address": "fa:16:3e:c0:fc:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapee1cfaf9-15", "ovs_interfaceid": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.compute.manager [req-0a3f9247-1ca8-45a9-8c61-78198d88886f req-0d0075f0-c3dd-40cf-94c6-db308436811e service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received event network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0a3f9247-1ca8-45a9-8c61-78198d88886f req-0d0075f0-c3dd-40cf-94c6-db308436811e service nova] Acquiring lock "de40e137-84e3-4d85-895b-1f6da2673457-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0a3f9247-1ca8-45a9-8c61-78198d88886f req-0d0075f0-c3dd-40cf-94c6-db308436811e service nova] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-0a3f9247-1ca8-45a9-8c61-78198d88886f req-0d0075f0-c3dd-40cf-94c6-db308436811e service nova] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.compute.manager [req-0a3f9247-1ca8-45a9-8c61-78198d88886f req-0d0075f0-c3dd-40cf-94c6-db308436811e service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] No waiting events found dispatching network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:58:11 user nova-compute[72081]: WARNING nova.compute.manager [req-0a3f9247-1ca8-45a9-8c61-78198d88886f req-0d0075f0-c3dd-40cf-94c6-db308436811e service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received unexpected event network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c for instance with vm_state building and task_state spawning.
Apr 12 03:58:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Releasing lock "refresh_cache-6cd92d70-a2af-4b2a-97c3-2981c157278d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Instance network_info: |[{"id": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "address": "fa:16:3e:c0:fc:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapee1cfaf9-15", "ovs_interfaceid": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1f4707e7-0b29-450d-bd3a-18eb5c6e59f6 req-fbacfbd0-0b06-40eb-b50a-20b5ed2a246c service nova] Acquired lock "refresh_cache-6cd92d70-a2af-4b2a-97c3-2981c157278d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.network.neutron [req-1f4707e7-0b29-450d-bd3a-18eb5c6e59f6 req-fbacfbd0-0b06-40eb-b50a-20b5ed2a246c service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Refreshing network info cache for port ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Start _get_guest_xml network_info=[{"id": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "address": "fa:16:3e:c0:fc:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapee1cfaf9-15", "ovs_interfaceid": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 03:58:11 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:58:11 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:58:07Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-965303931',display_name='tempest-ServerRescueNegativeTestJSON-server-965303931',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-965303931',id=45,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a6a8a4f5d59e479f9ca7b73a9e270c4b',ramdisk_id='',reservation_id='r-x2nt0nuo',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-529226261',owner_user_name='tempest-ServerRescueNegativeTestJSON-529226261-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:58:09Z,user_data=None,user_id='44a0206f1323440db4f8411a3c3c088d',uuid=6cd92d70-a2af-4b2a-97c3-2981c157278d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "address": "fa:16:3e:c0:fc:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapee1cfaf9-15", "ovs_interfaceid": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Converting VIF {"id": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "address": "fa:16:3e:c0:fc:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapee1cfaf9-15", "ovs_interfaceid": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:c0:fc:3b,bridge_name='br-int',has_traffic_filtering=True,id=ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda,network=Network(311bda05-075c-4bb3-9469-b5117dd33353),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapee1cfaf9-15') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'pci_devices' on Instance uuid 6cd92d70-a2af-4b2a-97c3-2981c157278d {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] End _get_guest_xml xml=<domain type="kvm">
Apr 12 03:58:11 user nova-compute[72081]:   <uuid>6cd92d70-a2af-4b2a-97c3-2981c157278d</uuid>
Apr 12 03:58:11 user nova-compute[72081]:   <name>instance-0000002d</name>
Apr 12 03:58:11 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 03:58:11 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 03:58:11 user nova-compute[72081]:   <metadata>
Apr 12 03:58:11 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 03:58:11 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 03:58:11 user nova-compute[72081]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-965303931</nova:name>
Apr 12 03:58:11 user nova-compute[72081]:       <nova:creationTime>2023-04-12 03:58:11</nova:creationTime>
Apr 12 03:58:11 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 03:58:11 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 03:58:11 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 03:58:11 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 03:58:11 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 03:58:11 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 03:58:11 user nova-compute[72081]:       </nova:flavor>
Apr 12 03:58:11 user nova-compute[72081]:       <nova:owner>
Apr 12 03:58:11 user nova-compute[72081]:         <nova:user uuid="44a0206f1323440db4f8411a3c3c088d">tempest-ServerRescueNegativeTestJSON-529226261-project-member</nova:user>
Apr 12 03:58:11 user nova-compute[72081]:         <nova:project uuid="a6a8a4f5d59e479f9ca7b73a9e270c4b">tempest-ServerRescueNegativeTestJSON-529226261</nova:project>
Apr 12 03:58:11 user nova-compute[72081]:       </nova:owner>
Apr 12 03:58:11 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 03:58:11 user nova-compute[72081]:       <nova:ports>
Apr 12 03:58:11 user nova-compute[72081]:         <nova:port uuid="ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda">
Apr 12 03:58:11 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.13" ipVersion="4"/>
Apr 12 03:58:11 user nova-compute[72081]:         </nova:port>
Apr 12 03:58:11 user nova-compute[72081]:       </nova:ports>
Apr 12 03:58:11 user nova-compute[72081]:     </nova:instance>
Apr 12 03:58:11 user nova-compute[72081]:   </metadata>
Apr 12 03:58:11 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 03:58:11 user nova-compute[72081]:     <system>
Apr 12 03:58:11 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 03:58:11 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 03:58:11 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 03:58:11 user nova-compute[72081]:       <entry name="serial">6cd92d70-a2af-4b2a-97c3-2981c157278d</entry>
Apr 12 03:58:11 user nova-compute[72081]:       <entry name="uuid">6cd92d70-a2af-4b2a-97c3-2981c157278d</entry>
Apr 12 03:58:11 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 03:58:11 user nova-compute[72081]:     </system>
Apr 12 03:58:11 user nova-compute[72081]:   </sysinfo>
Apr 12 03:58:11 user nova-compute[72081]:   <os>
Apr 12 03:58:11 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 03:58:11 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 03:58:11 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 03:58:11 user nova-compute[72081]:   </os>
Apr 12 03:58:11 user nova-compute[72081]:   <features>
Apr 12 03:58:11 user nova-compute[72081]:     <acpi/>
Apr 12 03:58:11 user nova-compute[72081]:     <apic/>
Apr 12 03:58:11 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 03:58:11 user nova-compute[72081]:   </features>
Apr 12 03:58:11 user nova-compute[72081]:   <clock offset="utc">
Apr 12 03:58:11 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 03:58:11 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 03:58:11 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 03:58:11 user nova-compute[72081]:   </clock>
Apr 12 03:58:11 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 03:58:11 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 03:58:11 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 03:58:11 user nova-compute[72081]:   </cpu>
Apr 12 03:58:11 user nova-compute[72081]:   <devices>
Apr 12 03:58:11 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 03:58:11 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 03:58:11 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk"/>
Apr 12 03:58:11 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 03:58:11 user nova-compute[72081]:     </disk>
Apr 12 03:58:11 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 03:58:11 user nova-compute[72081]:       <mac address="fa:16:3e:c0:fc:3b"/>
Apr 12 03:58:11 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:58:11 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 03:58:11 user nova-compute[72081]:       <target dev="tapee1cfaf9-15"/>
Apr 12 03:58:11 user nova-compute[72081]:     </interface>
Apr 12 03:58:11 user nova-compute[72081]:     <serial type="pty">
Apr 12 03:58:11 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/console.log" append="off"/>
Apr 12 03:58:11 user nova-compute[72081]:     </serial>
Apr 12 03:58:11 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 03:58:11 user nova-compute[72081]:     <video>
Apr 12 03:58:11 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 03:58:11 user nova-compute[72081]:     </video>
Apr 12 03:58:11 user nova-compute[72081]:     <rng model="virtio">
Apr 12 03:58:11 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 03:58:11 user nova-compute[72081]:     </rng>
Apr 12 03:58:11 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 03:58:11 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 03:58:11 user nova-compute[72081]:       <stats period="10"/>
Apr 12 03:58:11 user nova-compute[72081]:     </memballoon>
Apr 12 03:58:11 user nova-compute[72081]:   </devices>
Apr 12 03:58:11 user nova-compute[72081]: </domain>
Apr 12 03:58:11 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:58:07Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-965303931',display_name='tempest-ServerRescueNegativeTestJSON-server-965303931',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-965303931',id=45,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a6a8a4f5d59e479f9ca7b73a9e270c4b',ramdisk_id='',reservation_id='r-x2nt0nuo',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-529226261',owner_user_name='tempest-ServerRescueNegativeTestJSON-529226261-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:58:09Z,user_data=None,user_id='44a0206f1323440db4f8411a3c3c088d',uuid=6cd92d70-a2af-4b2a-97c3-2981c157278d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "address": "fa:16:3e:c0:fc:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapee1cfaf9-15", "ovs_interfaceid": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Converting VIF {"id": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "address": "fa:16:3e:c0:fc:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapee1cfaf9-15", "ovs_interfaceid": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:c0:fc:3b,bridge_name='br-int',has_traffic_filtering=True,id=ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda,network=Network(311bda05-075c-4bb3-9469-b5117dd33353),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapee1cfaf9-15') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG os_vif [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:c0:fc:3b,bridge_name='br-int',has_traffic_filtering=True,id=ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda,network=Network(311bda05-075c-4bb3-9469-b5117dd33353),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapee1cfaf9-15') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapee1cfaf9-15, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapee1cfaf9-15, col_values=(('external_ids', {'iface-id': 'ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:c0:fc:3b', 'vm-uuid': '6cd92d70-a2af-4b2a-97c3-2981c157278d'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:11 user nova-compute[72081]: INFO os_vif [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:c0:fc:3b,bridge_name='br-int',has_traffic_filtering=True,id=ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda,network=Network(311bda05-075c-4bb3-9469-b5117dd33353),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapee1cfaf9-15')
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] No VIF found with MAC fa:16:3e:c0:fc:3b, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:11 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:12 user nova-compute[72081]: DEBUG nova.network.neutron [req-1f4707e7-0b29-450d-bd3a-18eb5c6e59f6 req-fbacfbd0-0b06-40eb-b50a-20b5ed2a246c service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Updated VIF entry in instance network info cache for port ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:58:12 user nova-compute[72081]: DEBUG nova.network.neutron [req-1f4707e7-0b29-450d-bd3a-18eb5c6e59f6 req-fbacfbd0-0b06-40eb-b50a-20b5ed2a246c service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Updating instance_info_cache with network_info: [{"id": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "address": "fa:16:3e:c0:fc:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapee1cfaf9-15", "ovs_interfaceid": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:58:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1f4707e7-0b29-450d-bd3a-18eb5c6e59f6 req-fbacfbd0-0b06-40eb-b50a-20b5ed2a246c service nova] Releasing lock "refresh_cache-6cd92d70-a2af-4b2a-97c3-2981c157278d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:58:12 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681271877.3761926, faa9190f-a85c-4107-9c7c-3de959b5d4d0 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:58:12 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] VM Stopped (Lifecycle Event)
Apr 12 03:58:12 user nova-compute[72081]: DEBUG nova.compute.manager [None req-d9f7e4de-19ad-4ad3-97c5-015cafbfbf44 None None] [instance: faa9190f-a85c-4107-9c7c-3de959b5d4d0] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:58:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:12 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:13 user nova-compute[72081]: DEBUG nova.compute.manager [req-bef7bfe9-60c7-4ae4-9fb0-59a789349faf req-8d3f93ed-bc1a-46b7-97ed-c755045cf332 service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Received event network-vif-plugged-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:58:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bef7bfe9-60c7-4ae4-9fb0-59a789349faf req-8d3f93ed-bc1a-46b7-97ed-c755045cf332 service nova] Acquiring lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:58:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bef7bfe9-60c7-4ae4-9fb0-59a789349faf req-8d3f93ed-bc1a-46b7-97ed-c755045cf332 service nova] Lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:58:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-bef7bfe9-60c7-4ae4-9fb0-59a789349faf req-8d3f93ed-bc1a-46b7-97ed-c755045cf332 service nova] Lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:58:13 user nova-compute[72081]: DEBUG nova.compute.manager [req-bef7bfe9-60c7-4ae4-9fb0-59a789349faf req-8d3f93ed-bc1a-46b7-97ed-c755045cf332 service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] No waiting events found dispatching network-vif-plugged-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:58:13 user nova-compute[72081]: WARNING nova.compute.manager [req-bef7bfe9-60c7-4ae4-9fb0-59a789349faf req-8d3f93ed-bc1a-46b7-97ed-c755045cf332 service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Received unexpected event network-vif-plugged-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda for instance with vm_state building and task_state spawning.
Apr 12 03:58:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:13 user nova-compute[72081]: DEBUG nova.compute.manager [req-20fcce6c-feb4-4a42-b818-56dbf4ce9943 req-86fc04c3-13b8-4c3b-aacc-f52e888a7b8b service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received event network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:58:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-20fcce6c-feb4-4a42-b818-56dbf4ce9943 req-86fc04c3-13b8-4c3b-aacc-f52e888a7b8b service nova] Acquiring lock "de40e137-84e3-4d85-895b-1f6da2673457-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:58:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-20fcce6c-feb4-4a42-b818-56dbf4ce9943 req-86fc04c3-13b8-4c3b-aacc-f52e888a7b8b service nova] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:58:13 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-20fcce6c-feb4-4a42-b818-56dbf4ce9943 req-86fc04c3-13b8-4c3b-aacc-f52e888a7b8b service nova] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:58:13 user nova-compute[72081]: DEBUG nova.compute.manager [req-20fcce6c-feb4-4a42-b818-56dbf4ce9943 req-86fc04c3-13b8-4c3b-aacc-f52e888a7b8b service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] No waiting events found dispatching network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:58:13 user nova-compute[72081]: WARNING nova.compute.manager [req-20fcce6c-feb4-4a42-b818-56dbf4ce9943 req-86fc04c3-13b8-4c3b-aacc-f52e888a7b8b service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received unexpected event network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c for instance with vm_state building and task_state spawning.
Apr 12 03:58:14 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271894.280069, de40e137-84e3-4d85-895b-1f6da2673457 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:58:14 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] VM Resumed (Lifecycle Event)
Apr 12 03:58:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:58:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:58:14 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Instance spawned successfully.
Apr 12 03:58:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:58:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:58:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:58:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:58:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:58:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:58:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:58:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:58:14 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:58:14 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:58:14 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271894.28204, de40e137-84e3-4d85-895b-1f6da2673457 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:58:14 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] VM Started (Lifecycle Event)
Apr 12 03:58:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:58:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:58:14 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:58:14 user nova-compute[72081]: INFO nova.compute.manager [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Took 6.75 seconds to spawn the instance on the hypervisor.
Apr 12 03:58:14 user nova-compute[72081]: DEBUG nova.compute.manager [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:58:14 user nova-compute[72081]: INFO nova.compute.manager [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Took 7.29 seconds to build instance.
Apr 12 03:58:14 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-7a26269b-185f-4336-b8b2-198bdcd1815f tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "de40e137-84e3-4d85-895b-1f6da2673457" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.376s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:58:15 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271895.1489546, 6cd92d70-a2af-4b2a-97c3-2981c157278d => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:58:15 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] VM Resumed (Lifecycle Event)
Apr 12 03:58:15 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 03:58:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 03:58:15 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Instance spawned successfully.
Apr 12 03:58:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 03:58:15 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:58:15 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:58:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:58:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:58:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:58:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:58:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:58:15 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 03:58:15 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:58:15 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681271895.1504598, 6cd92d70-a2af-4b2a-97c3-2981c157278d => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:58:15 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] VM Started (Lifecycle Event)
Apr 12 03:58:15 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:58:15 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 03:58:15 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 03:58:15 user nova-compute[72081]: DEBUG nova.compute.manager [req-c9d1a834-2276-4d6a-97c0-dd92edb0a3b4 req-cba05c46-dcb9-4fb2-8275-64178dd7ef39 service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Received event network-vif-plugged-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:58:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c9d1a834-2276-4d6a-97c0-dd92edb0a3b4 req-cba05c46-dcb9-4fb2-8275-64178dd7ef39 service nova] Acquiring lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:58:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c9d1a834-2276-4d6a-97c0-dd92edb0a3b4 req-cba05c46-dcb9-4fb2-8275-64178dd7ef39 service nova] Lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:58:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c9d1a834-2276-4d6a-97c0-dd92edb0a3b4 req-cba05c46-dcb9-4fb2-8275-64178dd7ef39 service nova] Lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:58:15 user nova-compute[72081]: DEBUG nova.compute.manager [req-c9d1a834-2276-4d6a-97c0-dd92edb0a3b4 req-cba05c46-dcb9-4fb2-8275-64178dd7ef39 service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] No waiting events found dispatching network-vif-plugged-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:58:15 user nova-compute[72081]: WARNING nova.compute.manager [req-c9d1a834-2276-4d6a-97c0-dd92edb0a3b4 req-cba05c46-dcb9-4fb2-8275-64178dd7ef39 service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Received unexpected event network-vif-plugged-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda for instance with vm_state building and task_state spawning.
Apr 12 03:58:15 user nova-compute[72081]: INFO nova.compute.manager [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Took 6.56 seconds to spawn the instance on the hypervisor.
Apr 12 03:58:15 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:58:15 user nova-compute[72081]: INFO nova.compute.manager [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Took 7.24 seconds to build instance.
Apr 12 03:58:15 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4512c8b6-16fa-4ba7-9380-d4e5b2049fcf tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "6cd92d70-a2af-4b2a-97c3-2981c157278d" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.362s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:58:16 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:21 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:58:21 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:22 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:58:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:26 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:58:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:58:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:58:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:58:26 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:58:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:58:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:58:26 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:58:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.158s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:58:27 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 03:58:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:58:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:58:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:58:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:58:27 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:58:27 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:58:27 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8435MB free_disk=26.018367767333984GB free_vcpus=9 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:58:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:58:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:58:27 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 1e4dae15-f69f-4e44-959a-7480e185020b actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:58:27 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance de40e137-84e3-4d85-895b-1f6da2673457 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:58:27 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 6cd92d70-a2af-4b2a-97c3-2981c157278d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:58:27 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:58:27 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=896MB phys_disk=40GB used_disk=2GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:58:27 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:58:28 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:58:28 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:58:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.266s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:58:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:30 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:58:30 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:58:30 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:58:30 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:58:31 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:58:31 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:32 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:58:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:33 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:58:33 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:58:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:58:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:58:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:58:33 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 03:58:34 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Updating instance_info_cache with network_info: [{"id": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "address": "fa:16:3e:e9:ff:d5", "network": {"id": "99c0b9a9-e1b4-43d1-bbbb-66560eca330a", "bridge": "br-int", "label": "tempest-TestShelveInstance-2092145606-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.62", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d8ec2318ea05403db924630461ea77e5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be56e0c-9e", "ovs_interfaceid": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:58:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:58:34 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 03:58:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG nova.compute.manager [req-f9d45aea-c10c-484e-8d59-fad8c909e687 req-875f8379-1e69-4b69-af94-8008844daf70 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Received event network-changed-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG nova.compute.manager [req-f9d45aea-c10c-484e-8d59-fad8c909e687 req-875f8379-1e69-4b69-af94-8008844daf70 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Refreshing instance network info cache due to event network-changed-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f9d45aea-c10c-484e-8d59-fad8c909e687 req-875f8379-1e69-4b69-af94-8008844daf70 service nova] Acquiring lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f9d45aea-c10c-484e-8d59-fad8c909e687 req-875f8379-1e69-4b69-af94-8008844daf70 service nova] Acquired lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG nova.network.neutron [req-f9d45aea-c10c-484e-8d59-fad8c909e687 req-875f8379-1e69-4b69-af94-8008844daf70 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Refreshing network info cache for port 1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Acquiring lock "1e4dae15-f69f-4e44-959a-7480e185020b" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "1e4dae15-f69f-4e44-959a-7480e185020b" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Acquiring lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:58:44 user nova-compute[72081]: INFO nova.compute.manager [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Terminating instance
Apr 12 03:58:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG nova.compute.manager [req-72b4b8cc-5213-4192-9e90-fe00c80876cd req-010d2112-6d5b-481a-bc1a-587844857d46 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Received event network-vif-unplugged-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-72b4b8cc-5213-4192-9e90-fe00c80876cd req-010d2112-6d5b-481a-bc1a-587844857d46 service nova] Acquiring lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-72b4b8cc-5213-4192-9e90-fe00c80876cd req-010d2112-6d5b-481a-bc1a-587844857d46 service nova] Lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-72b4b8cc-5213-4192-9e90-fe00c80876cd req-010d2112-6d5b-481a-bc1a-587844857d46 service nova] Lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG nova.compute.manager [req-72b4b8cc-5213-4192-9e90-fe00c80876cd req-010d2112-6d5b-481a-bc1a-587844857d46 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] No waiting events found dispatching network-vif-unplugged-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG nova.compute.manager [req-72b4b8cc-5213-4192-9e90-fe00c80876cd req-010d2112-6d5b-481a-bc1a-587844857d46 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Received event network-vif-unplugged-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG nova.network.neutron [req-f9d45aea-c10c-484e-8d59-fad8c909e687 req-875f8379-1e69-4b69-af94-8008844daf70 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Updated VIF entry in instance network info cache for port 1be56e0c-9ee5-4df6-bbd6-f866a4e54e90. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG nova.network.neutron [req-f9d45aea-c10c-484e-8d59-fad8c909e687 req-875f8379-1e69-4b69-af94-8008844daf70 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Updating instance_info_cache with network_info: [{"id": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "address": "fa:16:3e:e9:ff:d5", "network": {"id": "99c0b9a9-e1b4-43d1-bbbb-66560eca330a", "bridge": "br-int", "label": "tempest-TestShelveInstance-2092145606-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d8ec2318ea05403db924630461ea77e5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be56e0c-9e", "ovs_interfaceid": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f9d45aea-c10c-484e-8d59-fad8c909e687 req-875f8379-1e69-4b69-af94-8008844daf70 service nova] Releasing lock "refresh_cache-1e4dae15-f69f-4e44-959a-7480e185020b" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:58:44 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Instance destroyed successfully.
Apr 12 03:58:44 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lazy-loading 'resources' on Instance uuid 1e4dae15-f69f-4e44-959a-7480e185020b {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-12T03:53:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestShelveInstance-server-1000809041',display_name='tempest-TestShelveInstance-server-1000809041',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testshelveinstance-server-1000809041',id=42,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHHBsGFyn1IfBui/0BXjt7hpTkTH4PsZRTzQ4VM6GFP5pgRYv1Yo7P5WOmhK4SKKnd1P8T0z7p8gKrx3Yxp/sOlxdxioj+WGXT88SaOKal87PVel/g0WojZ8NRP1p7DX/g==',key_name='tempest-TestShelveInstance-1986276884',keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:57:04Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='d8ec2318ea05403db924630461ea77e5',ramdisk_id='',reservation_id='r-3erg1hz6',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestShelveInstance-130328817',owner_user_name='tempest-TestShelveInstance-130328817-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T03:57:05Z,user_data=None,user_id='447437f9d71b4f8ab0da168f0fca732f',uuid=1e4dae15-f69f-4e44-959a-7480e185020b,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "address": "fa:16:3e:e9:ff:d5", "network": {"id": "99c0b9a9-e1b4-43d1-bbbb-66560eca330a", "bridge": "br-int", "label": "tempest-TestShelveInstance-2092145606-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.62", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d8ec2318ea05403db924630461ea77e5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be56e0c-9e", "ovs_interfaceid": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Converting VIF {"id": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "address": "fa:16:3e:e9:ff:d5", "network": {"id": "99c0b9a9-e1b4-43d1-bbbb-66560eca330a", "bridge": "br-int", "label": "tempest-TestShelveInstance-2092145606-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.62", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d8ec2318ea05403db924630461ea77e5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be56e0c-9e", "ovs_interfaceid": "1be56e0c-9ee5-4df6-bbd6-f866a4e54e90", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:e9:ff:d5,bridge_name='br-int',has_traffic_filtering=True,id=1be56e0c-9ee5-4df6-bbd6-f866a4e54e90,network=Network(99c0b9a9-e1b4-43d1-bbbb-66560eca330a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1be56e0c-9e') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG os_vif [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:e9:ff:d5,bridge_name='br-int',has_traffic_filtering=True,id=1be56e0c-9ee5-4df6-bbd6-f866a4e54e90,network=Network(99c0b9a9-e1b4-43d1-bbbb-66560eca330a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1be56e0c-9e') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap1be56e0c-9e, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:58:44 user nova-compute[72081]: INFO os_vif [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:e9:ff:d5,bridge_name='br-int',has_traffic_filtering=True,id=1be56e0c-9ee5-4df6-bbd6-f866a4e54e90,network=Network(99c0b9a9-e1b4-43d1-bbbb-66560eca330a),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1be56e0c-9e')
Apr 12 03:58:44 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 03:58:44 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OZAguYvD', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 03:58:44 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 03:58:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[152e31f9-d90d-4ea3-b06a-07c4e6c9feac]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 03:58:45 user nova-compute[72081]: No targets found.
Apr 12 03:58:45 user nova-compute[72081]: iSNS:
Apr 12 03:58:45 user nova-compute[72081]: No targets found.
Apr 12 03:58:45 user nova-compute[72081]: STATIC:
Apr 12 03:58:45 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02
Apr 12 03:58:45 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 03:58:45 user nova-compute[72081]:                 Iface Name: default
Apr 12 03:58:45 user nova-compute[72081]: FIRMWARE:
Apr 12 03:58:45 user nova-compute[72081]: No targets found.
Apr 12 03:58:45 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 03:58:45 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 03:58:45 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6863e6c8-3e8f-405a-9819-3a9d4b477466]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[43ea3043-944f-4d0c-85fb-c73c614f51ea]: (4, ('tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)
Apr 12 03:58:45 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsi session list stdout=tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 (non-flash)
Apr 12 03:58:45 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fba383250>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02'): ({'sdb'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Removing single pathed devices sdb {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bfc0794b-3ab7-4b9e-bcea-b3f11d1d2d18]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3a9f8304-5e10-45d5-be5d-92ae0902aa7d]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Flushing IO for device /dev/sdb {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c33ec43a-2110-4853-b9c9-905c25775d82]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.039s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e4532be4-4659-4f81-baba-108d7de40c6d]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] SCSI volumes sdb have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-23564643731633831', '/dev/disk/by-id/scsi-SSCST_BIO_Storage_cba8206a', '/dev/disk/by-id/scsi-1SCST_BIOcba8206a-5dd71c8130b01756') {{(pid=82083) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Apr 12 03:58:45 user nova-compute[72081]: WARNING os_brick.exception [-] Unlink failed for /dev/disk/by-id/scsi-23564643731633831
Apr 12 03:58:45 user nova-compute[72081]: WARNING os_brick.exception [-] Unlink failed for /dev/disk/by-id/scsi-SSCST_BIO_Storage_cba8206a
Apr 12 03:58:45 user nova-compute[72081]: WARNING os_brick.exception [-] Unlink failed for /dev/disk/by-id/scsi-1SCST_BIOcba8206a-5dd71c8130b01756
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0ecd3760-bb77-4bf6-822f-9ebdb55d8aab]: (4, None) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.036s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b692c95f-07b6-4281-a40e-bc54b1ea2b3e]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --logout" returned: 0 in 0.044s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[24fecd91-12b0-4330-832f-d14b8c17ae55]: (4, ('Logging out of session [sid: 49, target: iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02, portal: 172.16.0.220,3260]\nLogout of [sid: 49, target: iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 49, target: iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02, portal: 172.16.0.220,3260]
Apr 12 03:58:45 user nova-compute[72081]: Logout of [sid: 49, target: iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02, portal: 172.16.0.220,3260] successful.
Apr 12 03:58:45 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d6a07e3a-be2a-4191-b1b7-09667064fb02 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2737b71c-53f4-4c41-aa83-6af7f25ce4b9]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.313s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] <== disconnect_volume: return (314ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 03:58:45 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Deleting instance files /opt/stack/data/nova/instances/1e4dae15-f69f-4e44-959a-7480e185020b_del
Apr 12 03:58:45 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Deletion of /opt/stack/data/nova/instances/1e4dae15-f69f-4e44-959a-7480e185020b_del complete
Apr 12 03:58:45 user nova-compute[72081]: INFO nova.compute.manager [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Took 1.16 seconds to destroy the instance on the hypervisor.
Apr 12 03:58:45 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 03:58:45 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:58:45 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Took 0.44 seconds to deallocate network for instance.
Apr 12 03:58:46 user nova-compute[72081]: DEBUG nova.compute.manager [req-0a381734-0f76-4094-98c5-2dfdd3a15128 req-d02496de-82fb-485a-91d5-7a3a385812c1 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Received event network-vif-deleted-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:58:46 user nova-compute[72081]: DEBUG nova.compute.manager [req-6aa51772-1356-46cd-9bc6-11e79da80955 req-2ebc5732-a6e5-4557-8e1e-44a7a024bef3 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Received event network-vif-plugged-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:58:46 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6aa51772-1356-46cd-9bc6-11e79da80955 req-2ebc5732-a6e5-4557-8e1e-44a7a024bef3 service nova] Acquiring lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:58:46 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6aa51772-1356-46cd-9bc6-11e79da80955 req-2ebc5732-a6e5-4557-8e1e-44a7a024bef3 service nova] Lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:58:46 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-6aa51772-1356-46cd-9bc6-11e79da80955 req-2ebc5732-a6e5-4557-8e1e-44a7a024bef3 service nova] Lock "1e4dae15-f69f-4e44-959a-7480e185020b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:58:46 user nova-compute[72081]: DEBUG nova.compute.manager [req-6aa51772-1356-46cd-9bc6-11e79da80955 req-2ebc5732-a6e5-4557-8e1e-44a7a024bef3 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] No waiting events found dispatching network-vif-plugged-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:58:46 user nova-compute[72081]: WARNING nova.compute.manager [req-6aa51772-1356-46cd-9bc6-11e79da80955 req-2ebc5732-a6e5-4557-8e1e-44a7a024bef3 service nova] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Received unexpected event network-vif-plugged-1be56e0c-9ee5-4df6-bbd6-f866a4e54e90 for instance with vm_state active and task_state deleting.
Apr 12 03:58:46 user nova-compute[72081]: INFO nova.compute.manager [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Took 0.96 seconds to detach 1 volumes for instance.
Apr 12 03:58:46 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Deleting volume: d6a07e3a-be2a-4191-b1b7-09667064fb02 {{(pid=72081) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3202}}
Apr 12 03:58:46 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:58:46 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:58:47 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:58:47 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:58:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.165s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:58:47 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Deleted allocations for instance 1e4dae15-f69f-4e44-959a-7480e185020b
Apr 12 03:58:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4bae6da8-fb96-4193-9840-75b66585cacb tempest-TestShelveInstance-130328817 tempest-TestShelveInstance-130328817-project-member] Lock "1e4dae15-f69f-4e44-959a-7480e185020b" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.003s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:58:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:58:59 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681271924.9575334, 1e4dae15-f69f-4e44-959a-7480e185020b => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 03:58:59 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] VM Stopped (Lifecycle Event)
Apr 12 03:58:59 user nova-compute[72081]: DEBUG nova.compute.manager [None req-d19b35a7-2532-44e0-9c90-ee887862255b None None] [instance: 1e4dae15-f69f-4e44-959a-7480e185020b] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 03:58:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:06 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:21 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:59:24 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:59:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:27 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:59:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:59:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:59:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:59:27 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 03:59:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:59:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:59:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:59:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:59:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:59:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:59:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:59:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.158s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:59:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:28 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:59:28 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 03:59:28 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8614MB free_disk=25.99715805053711GB free_vcpus=10 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 03:59:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:59:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:59:28 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance de40e137-84e3-4d85-895b-1f6da2673457 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:59:28 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 6cd92d70-a2af-4b2a-97c3-2981c157278d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 03:59:28 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 2 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 03:59:28 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=768MB phys_disk=40GB used_disk=2GB total_vcpus=12 used_vcpus=2 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 03:59:28 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:59:28 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:59:28 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 03:59:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.251s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:59:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:31 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:59:31 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:59:31 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:59:31 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:59:31 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 03:59:32 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:59:34 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 03:59:34 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 03:59:34 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Rebuilding the list of instances to heal {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 12 03:59:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:59:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:59:34 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 03:59:34 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lazy-loading 'info_cache' on Instance uuid de40e137-84e3-4d85-895b-1f6da2673457 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:59:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:59:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 03:59:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 03:59:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 03:59:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:35 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Updating instance_info_cache with network_info: [{"id": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "address": "fa:16:3e:c4:e2:d7", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap16a3f234-3d", "ovs_interfaceid": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:59:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:59:35 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 03:59:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:45 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 03:59:45 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:47 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:59:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:59:57 user nova-compute[72081]: DEBUG nova.compute.manager [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 03:59:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:59:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:59:57 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 03:59:57 user nova-compute[72081]: INFO nova.compute.claims [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Claim successful on node user
Apr 12 03:59:57 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 03:59:57 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 03:59:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.245s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:59:57 user nova-compute[72081]: DEBUG nova.compute.manager [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 03:59:57 user nova-compute[72081]: DEBUG nova.compute.manager [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 03:59:57 user nova-compute[72081]: DEBUG nova.network.neutron [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 03:59:57 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 03:59:57 user nova-compute[72081]: DEBUG nova.compute.manager [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 03:59:57 user nova-compute[72081]: DEBUG nova.policy [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'dc05e2e474814697920e72d50883e263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01223543586e474b875e7dbf2c78ef87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 03:59:57 user nova-compute[72081]: DEBUG nova.compute.manager [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 03:59:57 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 03:59:57 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Creating image(s)
Apr 12 03:59:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "/opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:59:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "/opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:59:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "/opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.007s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:59:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:59:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.131s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:59:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:59:57 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:59:57 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:59:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.150s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:59:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:59:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk 1073741824" returned: 0 in 0.073s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:59:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.230s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:59:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:59:58 user nova-compute[72081]: INFO nova.compute.manager [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Rescuing
Apr 12 03:59:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "refresh_cache-6cd92d70-a2af-4b2a-97c3-2981c157278d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 03:59:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquired lock "refresh_cache-6cd92d70-a2af-4b2a-97c3-2981c157278d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 03:59:58 user nova-compute[72081]: DEBUG nova.network.neutron [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 03:59:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.165s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:59:58 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Checking if we can resize image /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 03:59:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 03:59:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 03:59:58 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Cannot resize image /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 03:59:58 user nova-compute[72081]: DEBUG nova.objects.instance [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lazy-loading 'migration_context' on Instance uuid 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 03:59:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 03:59:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Ensure instance console log exists: /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 03:59:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:59:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:59:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:59:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:59 user nova-compute[72081]: DEBUG nova.network.neutron [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Successfully created port: c83728df-a614-42d7-a164-d5d5e0373838 {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 03:59:59 user nova-compute[72081]: DEBUG nova.network.neutron [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Updating instance_info_cache with network_info: [{"id": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "address": "fa:16:3e:c0:fc:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapee1cfaf9-15", "ovs_interfaceid": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 03:59:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Releasing lock "refresh_cache-6cd92d70-a2af-4b2a-97c3-2981c157278d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 03:59:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-f4e935df-bb58-48eb-be18-4500e5f9305d req-f5f3bf5d-ebed-4c98-8ca7-5d34840b85ce service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Received event network-vif-unplugged-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 03:59:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f4e935df-bb58-48eb-be18-4500e5f9305d req-f5f3bf5d-ebed-4c98-8ca7-5d34840b85ce service nova] Acquiring lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 03:59:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f4e935df-bb58-48eb-be18-4500e5f9305d req-f5f3bf5d-ebed-4c98-8ca7-5d34840b85ce service nova] Lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 03:59:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f4e935df-bb58-48eb-be18-4500e5f9305d req-f5f3bf5d-ebed-4c98-8ca7-5d34840b85ce service nova] Lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 03:59:59 user nova-compute[72081]: DEBUG nova.compute.manager [req-f4e935df-bb58-48eb-be18-4500e5f9305d req-f5f3bf5d-ebed-4c98-8ca7-5d34840b85ce service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] No waiting events found dispatching network-vif-unplugged-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 03:59:59 user nova-compute[72081]: WARNING nova.compute.manager [req-f4e935df-bb58-48eb-be18-4500e5f9305d req-f5f3bf5d-ebed-4c98-8ca7-5d34840b85ce service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Received unexpected event network-vif-unplugged-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda for instance with vm_state active and task_state rescuing.
Apr 12 03:59:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 03:59:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.network.neutron [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Successfully updated port: c83728df-a614-42d7-a164-d5d5e0373838 {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "refresh_cache-7a6fb26c-9c8f-4be7-b16f-8662eed2e489" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquired lock "refresh_cache-7a6fb26c-9c8f-4be7-b16f-8662eed2e489" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.network.neutron [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.compute.manager [req-d0f4e2a9-a59d-4634-92e3-2c312584b22d req-8ac3518f-2c02-4024-9a21-6a0889ccc596 service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Received event network-changed-c83728df-a614-42d7-a164-d5d5e0373838 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.compute.manager [req-d0f4e2a9-a59d-4634-92e3-2c312584b22d req-8ac3518f-2c02-4024-9a21-6a0889ccc596 service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Refreshing instance network info cache due to event network-changed-c83728df-a614-42d7-a164-d5d5e0373838. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d0f4e2a9-a59d-4634-92e3-2c312584b22d req-8ac3518f-2c02-4024-9a21-6a0889ccc596 service nova] Acquiring lock "refresh_cache-7a6fb26c-9c8f-4be7-b16f-8662eed2e489" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.network.neutron [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 04:00:00 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Instance destroyed successfully.
Apr 12 04:00:00 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Attempting rescue
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} {{(pid=72081) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4289}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Instance directory exists: not creating {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4694}}
Apr 12 04:00:00 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Creating image(s)
Apr 12 04:00:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "/opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "/opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "/opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.021s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.objects.instance [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'trusted_certs' on Instance uuid 6cd92d70-a2af-4b2a-97c3-2981c157278d {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue" returned: 0 in 0.053s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.194s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.objects.instance [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'migration_context' on Instance uuid 6cd92d70-a2af-4b2a-97c3-2981c157278d {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Start _get_guest_xml network_info=[{"id": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "address": "fa:16:3e:c0:fc:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "vif_mac": "fa:16:3e:c0:fc:3b"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapee1cfaf9-15", "ovs_interfaceid": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7', 'kernel_id': '', 'ramdisk_id': ''} block_device_info=None {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.objects.instance [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'resources' on Instance uuid 6cd92d70-a2af-4b2a-97c3-2981c157278d {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.network.neutron [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Updating instance_info_cache with network_info: [{"id": "c83728df-a614-42d7-a164-d5d5e0373838", "address": "fa:16:3e:b8:39:b0", "network": {"id": "1298d27c-9dfa-46e2-b941-024dd99ead79", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-1228939766-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01223543586e474b875e7dbf2c78ef87", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc83728df-a6", "ovs_interfaceid": "c83728df-a614-42d7-a164-d5d5e0373838", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.objects.instance [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'numa_topology' on Instance uuid 6cd92d70-a2af-4b2a-97c3-2981c157278d {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Releasing lock "refresh_cache-7a6fb26c-9c8f-4be7-b16f-8662eed2e489" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.compute.manager [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Instance network_info: |[{"id": "c83728df-a614-42d7-a164-d5d5e0373838", "address": "fa:16:3e:b8:39:b0", "network": {"id": "1298d27c-9dfa-46e2-b941-024dd99ead79", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-1228939766-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01223543586e474b875e7dbf2c78ef87", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc83728df-a6", "ovs_interfaceid": "c83728df-a614-42d7-a164-d5d5e0373838", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d0f4e2a9-a59d-4634-92e3-2c312584b22d req-8ac3518f-2c02-4024-9a21-6a0889ccc596 service nova] Acquired lock "refresh_cache-7a6fb26c-9c8f-4be7-b16f-8662eed2e489" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.network.neutron [req-d0f4e2a9-a59d-4634-92e3-2c312584b22d req-8ac3518f-2c02-4024-9a21-6a0889ccc596 service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Refreshing network info cache for port c83728df-a614-42d7-a164-d5d5e0373838 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Start _get_guest_xml network_info=[{"id": "c83728df-a614-42d7-a164-d5d5e0373838", "address": "fa:16:3e:b8:39:b0", "network": {"id": "1298d27c-9dfa-46e2-b941-024dd99ead79", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-1228939766-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01223543586e474b875e7dbf2c78ef87", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc83728df-a6", "ovs_interfaceid": "c83728df-a614-42d7-a164-d5d5e0373838", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 04:00:00 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:00:00 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.objects.instance [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'vcpu_model' on Instance uuid 6cd92d70-a2af-4b2a-97c3-2981c157278d {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:00:00 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:00:00 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:59:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-SnapshotDataIntegrityTests-server-1263744278',display_name='tempest-SnapshotDataIntegrityTests-server-1263744278',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-snapshotdataintegritytests-server-1263744278',id=46,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS5zNm2ebJK6LG9OrL+ojzX+EwxzS/Yi0eAvnaL94lZQddER4Le1QtAFpoIWWwQszoaOHdLJaBYOeLou7nEAtCK/7ohtIg1OU5j5fJMB0C0xARTUV1WXA8kI8LAoQVCtA==',key_name='tempest-SnapshotDataIntegrityTests-272659456',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='01223543586e474b875e7dbf2c78ef87',ramdisk_id='',reservation_id='r-6z106nur',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-SnapshotDataIntegrityTests-956619934',owner_user_name='tempest-SnapshotDataIntegrityTests-956619934-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:59:58Z,user_data=None,user_id='dc05e2e474814697920e72d50883e263',uuid=7a6fb26c-9c8f-4be7-b16f-8662eed2e489,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c83728df-a614-42d7-a164-d5d5e0373838", "address": "fa:16:3e:b8:39:b0", "network": {"id": "1298d27c-9dfa-46e2-b941-024dd99ead79", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-1228939766-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01223543586e474b875e7dbf2c78ef87", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc83728df-a6", "ovs_interfaceid": "c83728df-a614-42d7-a164-d5d5e0373838", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Converting VIF {"id": "c83728df-a614-42d7-a164-d5d5e0373838", "address": "fa:16:3e:b8:39:b0", "network": {"id": "1298d27c-9dfa-46e2-b941-024dd99ead79", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-1228939766-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01223543586e474b875e7dbf2c78ef87", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc83728df-a6", "ovs_interfaceid": "c83728df-a614-42d7-a164-d5d5e0373838", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b8:39:b0,bridge_name='br-int',has_traffic_filtering=True,id=c83728df-a614-42d7-a164-d5d5e0373838,network=Network(1298d27c-9dfa-46e2-b941-024dd99ead79),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc83728df-a6') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.objects.instance [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lazy-loading 'pci_devices' on Instance uuid 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:58:07Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-965303931',display_name='tempest-ServerRescueNegativeTestJSON-server-965303931',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-965303931',id=45,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:58:15Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='a6a8a4f5d59e479f9ca7b73a9e270c4b',ramdisk_id='',reservation_id='r-x2nt0nuo',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-529226261',owner_user_name='tempest-ServerRescueNegativeTestJSON-529226261-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:58:15Z,user_data=None,user_id='44a0206f1323440db4f8411a3c3c088d',uuid=6cd92d70-a2af-4b2a-97c3-2981c157278d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "address": "fa:16:3e:c0:fc:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "vif_mac": "fa:16:3e:c0:fc:3b"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapee1cfaf9-15", "ovs_interfaceid": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Converting VIF {"id": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "address": "fa:16:3e:c0:fc:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "vif_mac": "fa:16:3e:c0:fc:3b"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapee1cfaf9-15", "ovs_interfaceid": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:c0:fc:3b,bridge_name='br-int',has_traffic_filtering=True,id=ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda,network=Network(311bda05-075c-4bb3-9469-b5117dd33353),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapee1cfaf9-15') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.objects.instance [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'pci_devices' on Instance uuid 6cd92d70-a2af-4b2a-97c3-2981c157278d {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] End _get_guest_xml xml=<domain type="kvm">
Apr 12 04:00:00 user nova-compute[72081]:   <uuid>7a6fb26c-9c8f-4be7-b16f-8662eed2e489</uuid>
Apr 12 04:00:00 user nova-compute[72081]:   <name>instance-0000002e</name>
Apr 12 04:00:00 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 04:00:00 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 04:00:00 user nova-compute[72081]:   <metadata>
Apr 12 04:00:00 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 04:00:00 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 04:00:00 user nova-compute[72081]:       <nova:name>tempest-SnapshotDataIntegrityTests-server-1263744278</nova:name>
Apr 12 04:00:00 user nova-compute[72081]:       <nova:creationTime>2023-04-12 04:00:00</nova:creationTime>
Apr 12 04:00:00 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 04:00:00 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 04:00:00 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 04:00:00 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 04:00:00 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 04:00:00 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 04:00:00 user nova-compute[72081]:       </nova:flavor>
Apr 12 04:00:00 user nova-compute[72081]:       <nova:owner>
Apr 12 04:00:00 user nova-compute[72081]:         <nova:user uuid="dc05e2e474814697920e72d50883e263">tempest-SnapshotDataIntegrityTests-956619934-project-member</nova:user>
Apr 12 04:00:00 user nova-compute[72081]:         <nova:project uuid="01223543586e474b875e7dbf2c78ef87">tempest-SnapshotDataIntegrityTests-956619934</nova:project>
Apr 12 04:00:00 user nova-compute[72081]:       </nova:owner>
Apr 12 04:00:00 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 04:00:00 user nova-compute[72081]:       <nova:ports>
Apr 12 04:00:00 user nova-compute[72081]:         <nova:port uuid="c83728df-a614-42d7-a164-d5d5e0373838">
Apr 12 04:00:00 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.8" ipVersion="4"/>
Apr 12 04:00:00 user nova-compute[72081]:         </nova:port>
Apr 12 04:00:00 user nova-compute[72081]:       </nova:ports>
Apr 12 04:00:00 user nova-compute[72081]:     </nova:instance>
Apr 12 04:00:00 user nova-compute[72081]:   </metadata>
Apr 12 04:00:00 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 04:00:00 user nova-compute[72081]:     <system>
Apr 12 04:00:00 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 04:00:00 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 04:00:00 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 04:00:00 user nova-compute[72081]:       <entry name="serial">7a6fb26c-9c8f-4be7-b16f-8662eed2e489</entry>
Apr 12 04:00:00 user nova-compute[72081]:       <entry name="uuid">7a6fb26c-9c8f-4be7-b16f-8662eed2e489</entry>
Apr 12 04:00:00 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 04:00:00 user nova-compute[72081]:     </system>
Apr 12 04:00:00 user nova-compute[72081]:   </sysinfo>
Apr 12 04:00:00 user nova-compute[72081]:   <os>
Apr 12 04:00:00 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 04:00:00 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 04:00:00 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 04:00:00 user nova-compute[72081]:   </os>
Apr 12 04:00:00 user nova-compute[72081]:   <features>
Apr 12 04:00:00 user nova-compute[72081]:     <acpi/>
Apr 12 04:00:00 user nova-compute[72081]:     <apic/>
Apr 12 04:00:00 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 04:00:00 user nova-compute[72081]:   </features>
Apr 12 04:00:00 user nova-compute[72081]:   <clock offset="utc">
Apr 12 04:00:00 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 04:00:00 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 04:00:00 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 04:00:00 user nova-compute[72081]:   </clock>
Apr 12 04:00:00 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 04:00:00 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 04:00:00 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 04:00:00 user nova-compute[72081]:   </cpu>
Apr 12 04:00:00 user nova-compute[72081]:   <devices>
Apr 12 04:00:00 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 04:00:00 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 04:00:00 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk"/>
Apr 12 04:00:00 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 04:00:00 user nova-compute[72081]:     </disk>
Apr 12 04:00:00 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 04:00:00 user nova-compute[72081]:       <mac address="fa:16:3e:b8:39:b0"/>
Apr 12 04:00:00 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 04:00:00 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 04:00:00 user nova-compute[72081]:       <target dev="tapc83728df-a6"/>
Apr 12 04:00:00 user nova-compute[72081]:     </interface>
Apr 12 04:00:00 user nova-compute[72081]:     <serial type="pty">
Apr 12 04:00:00 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/console.log" append="off"/>
Apr 12 04:00:00 user nova-compute[72081]:     </serial>
Apr 12 04:00:00 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 04:00:00 user nova-compute[72081]:     <video>
Apr 12 04:00:00 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 04:00:00 user nova-compute[72081]:     </video>
Apr 12 04:00:00 user nova-compute[72081]:     <rng model="virtio">
Apr 12 04:00:00 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 04:00:00 user nova-compute[72081]:     </rng>
Apr 12 04:00:00 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 04:00:00 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 04:00:00 user nova-compute[72081]:       <stats period="10"/>
Apr 12 04:00:00 user nova-compute[72081]:     </memballoon>
Apr 12 04:00:00 user nova-compute[72081]:   </devices>
Apr 12 04:00:00 user nova-compute[72081]: </domain>
Apr 12 04:00:00 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:59:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-SnapshotDataIntegrityTests-server-1263744278',display_name='tempest-SnapshotDataIntegrityTests-server-1263744278',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-snapshotdataintegritytests-server-1263744278',id=46,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS5zNm2ebJK6LG9OrL+ojzX+EwxzS/Yi0eAvnaL94lZQddER4Le1QtAFpoIWWwQszoaOHdLJaBYOeLou7nEAtCK/7ohtIg1OU5j5fJMB0C0xARTUV1WXA8kI8LAoQVCtA==',key_name='tempest-SnapshotDataIntegrityTests-272659456',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='01223543586e474b875e7dbf2c78ef87',ramdisk_id='',reservation_id='r-6z106nur',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-SnapshotDataIntegrityTests-956619934',owner_user_name='tempest-SnapshotDataIntegrityTests-956619934-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:59:58Z,user_data=None,user_id='dc05e2e474814697920e72d50883e263',uuid=7a6fb26c-9c8f-4be7-b16f-8662eed2e489,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c83728df-a614-42d7-a164-d5d5e0373838", "address": "fa:16:3e:b8:39:b0", "network": {"id": "1298d27c-9dfa-46e2-b941-024dd99ead79", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-1228939766-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01223543586e474b875e7dbf2c78ef87", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc83728df-a6", "ovs_interfaceid": "c83728df-a614-42d7-a164-d5d5e0373838", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Converting VIF {"id": "c83728df-a614-42d7-a164-d5d5e0373838", "address": "fa:16:3e:b8:39:b0", "network": {"id": "1298d27c-9dfa-46e2-b941-024dd99ead79", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-1228939766-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01223543586e474b875e7dbf2c78ef87", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc83728df-a6", "ovs_interfaceid": "c83728df-a614-42d7-a164-d5d5e0373838", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b8:39:b0,bridge_name='br-int',has_traffic_filtering=True,id=c83728df-a614-42d7-a164-d5d5e0373838,network=Network(1298d27c-9dfa-46e2-b941-024dd99ead79),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc83728df-a6') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG os_vif [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:b8:39:b0,bridge_name='br-int',has_traffic_filtering=True,id=c83728df-a614-42d7-a164-d5d5e0373838,network=Network(1298d27c-9dfa-46e2-b941-024dd99ead79),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc83728df-a6') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] End _get_guest_xml xml=<domain type="kvm">
Apr 12 04:00:00 user nova-compute[72081]:   <uuid>6cd92d70-a2af-4b2a-97c3-2981c157278d</uuid>
Apr 12 04:00:00 user nova-compute[72081]:   <name>instance-0000002d</name>
Apr 12 04:00:00 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 04:00:00 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 04:00:00 user nova-compute[72081]:   <metadata>
Apr 12 04:00:00 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 04:00:00 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 04:00:00 user nova-compute[72081]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-965303931</nova:name>
Apr 12 04:00:00 user nova-compute[72081]:       <nova:creationTime>2023-04-12 04:00:00</nova:creationTime>
Apr 12 04:00:00 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 04:00:00 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 04:00:00 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 04:00:00 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 04:00:00 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 04:00:00 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 04:00:00 user nova-compute[72081]:       </nova:flavor>
Apr 12 04:00:00 user nova-compute[72081]:       <nova:owner>
Apr 12 04:00:00 user nova-compute[72081]:         <nova:user uuid="44a0206f1323440db4f8411a3c3c088d">tempest-ServerRescueNegativeTestJSON-529226261-project-member</nova:user>
Apr 12 04:00:00 user nova-compute[72081]:         <nova:project uuid="a6a8a4f5d59e479f9ca7b73a9e270c4b">tempest-ServerRescueNegativeTestJSON-529226261</nova:project>
Apr 12 04:00:00 user nova-compute[72081]:       </nova:owner>
Apr 12 04:00:00 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 04:00:00 user nova-compute[72081]:       <nova:ports>
Apr 12 04:00:00 user nova-compute[72081]:         <nova:port uuid="ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda">
Apr 12 04:00:00 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.13" ipVersion="4"/>
Apr 12 04:00:00 user nova-compute[72081]:         </nova:port>
Apr 12 04:00:00 user nova-compute[72081]:       </nova:ports>
Apr 12 04:00:00 user nova-compute[72081]:     </nova:instance>
Apr 12 04:00:00 user nova-compute[72081]:   </metadata>
Apr 12 04:00:00 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 04:00:00 user nova-compute[72081]:     <system>
Apr 12 04:00:00 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 04:00:00 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 04:00:00 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 04:00:00 user nova-compute[72081]:       <entry name="serial">6cd92d70-a2af-4b2a-97c3-2981c157278d</entry>
Apr 12 04:00:00 user nova-compute[72081]:       <entry name="uuid">6cd92d70-a2af-4b2a-97c3-2981c157278d</entry>
Apr 12 04:00:00 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 04:00:00 user nova-compute[72081]:     </system>
Apr 12 04:00:00 user nova-compute[72081]:   </sysinfo>
Apr 12 04:00:00 user nova-compute[72081]:   <os>
Apr 12 04:00:00 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 04:00:00 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 04:00:00 user nova-compute[72081]:   </os>
Apr 12 04:00:00 user nova-compute[72081]:   <features>
Apr 12 04:00:00 user nova-compute[72081]:     <acpi/>
Apr 12 04:00:00 user nova-compute[72081]:     <apic/>
Apr 12 04:00:00 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 04:00:00 user nova-compute[72081]:   </features>
Apr 12 04:00:00 user nova-compute[72081]:   <clock offset="utc">
Apr 12 04:00:00 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 04:00:00 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 04:00:00 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 04:00:00 user nova-compute[72081]:   </clock>
Apr 12 04:00:00 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 04:00:00 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 04:00:00 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 04:00:00 user nova-compute[72081]:   </cpu>
Apr 12 04:00:00 user nova-compute[72081]:   <devices>
Apr 12 04:00:00 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 04:00:00 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 04:00:00 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue"/>
Apr 12 04:00:00 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 04:00:00 user nova-compute[72081]:     </disk>
Apr 12 04:00:00 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 04:00:00 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 04:00:00 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk"/>
Apr 12 04:00:00 user nova-compute[72081]:       <target dev="vdb" bus="virtio"/>
Apr 12 04:00:00 user nova-compute[72081]:     </disk>
Apr 12 04:00:00 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 04:00:00 user nova-compute[72081]:       <mac address="fa:16:3e:c0:fc:3b"/>
Apr 12 04:00:00 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 04:00:00 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 04:00:00 user nova-compute[72081]:       <target dev="tapee1cfaf9-15"/>
Apr 12 04:00:00 user nova-compute[72081]:     </interface>
Apr 12 04:00:00 user nova-compute[72081]:     <serial type="pty">
Apr 12 04:00:00 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/console.log" append="off"/>
Apr 12 04:00:00 user nova-compute[72081]:     </serial>
Apr 12 04:00:00 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 04:00:00 user nova-compute[72081]:     <video>
Apr 12 04:00:00 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 04:00:00 user nova-compute[72081]:     </video>
Apr 12 04:00:00 user nova-compute[72081]:     <rng model="virtio">
Apr 12 04:00:00 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 04:00:00 user nova-compute[72081]:     </rng>
Apr 12 04:00:00 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 04:00:00 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 04:00:00 user nova-compute[72081]:       <stats period="10"/>
Apr 12 04:00:00 user nova-compute[72081]:     </memballoon>
Apr 12 04:00:00 user nova-compute[72081]:   </devices>
Apr 12 04:00:00 user nova-compute[72081]: </domain>
Apr 12 04:00:00 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapc83728df-a6, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapc83728df-a6, col_values=(('external_ids', {'iface-id': 'c83728df-a614-42d7-a164-d5d5e0373838', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:b8:39:b0', 'vm-uuid': '7a6fb26c-9c8f-4be7-b16f-8662eed2e489'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:00:00 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Instance destroyed successfully.
Apr 12 04:00:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:00 user nova-compute[72081]: INFO os_vif [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:b8:39:b0,bridge_name='br-int',has_traffic_filtering=True,id=c83728df-a614-42d7-a164-d5d5e0373838,network=Network(1298d27c-9dfa-46e2-b941-024dd99ead79),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc83728df-a6')
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] No VIF found with MAC fa:16:3e:c0:fc:3b, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] No VIF found with MAC fa:16:3e:b8:39:b0, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.network.neutron [req-d0f4e2a9-a59d-4634-92e3-2c312584b22d req-8ac3518f-2c02-4024-9a21-6a0889ccc596 service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Updated VIF entry in instance network info cache for port c83728df-a614-42d7-a164-d5d5e0373838. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG nova.network.neutron [req-d0f4e2a9-a59d-4634-92e3-2c312584b22d req-8ac3518f-2c02-4024-9a21-6a0889ccc596 service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Updating instance_info_cache with network_info: [{"id": "c83728df-a614-42d7-a164-d5d5e0373838", "address": "fa:16:3e:b8:39:b0", "network": {"id": "1298d27c-9dfa-46e2-b941-024dd99ead79", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-1228939766-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01223543586e474b875e7dbf2c78ef87", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc83728df-a6", "ovs_interfaceid": "c83728df-a614-42d7-a164-d5d5e0373838", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:00:00 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d0f4e2a9-a59d-4634-92e3-2c312584b22d req-8ac3518f-2c02-4024-9a21-6a0889ccc596 service nova] Releasing lock "refresh_cache-7a6fb26c-9c8f-4be7-b16f-8662eed2e489" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:00:01 user nova-compute[72081]: DEBUG nova.compute.manager [req-995569ff-6838-4ac6-a3cc-036baa3a5340 req-de0d60fd-1b8f-4397-b126-0c8dcc5deb93 service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Received event network-vif-plugged-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:00:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-995569ff-6838-4ac6-a3cc-036baa3a5340 req-de0d60fd-1b8f-4397-b126-0c8dcc5deb93 service nova] Acquiring lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:00:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-995569ff-6838-4ac6-a3cc-036baa3a5340 req-de0d60fd-1b8f-4397-b126-0c8dcc5deb93 service nova] Lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:00:01 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-995569ff-6838-4ac6-a3cc-036baa3a5340 req-de0d60fd-1b8f-4397-b126-0c8dcc5deb93 service nova] Lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:00:01 user nova-compute[72081]: DEBUG nova.compute.manager [req-995569ff-6838-4ac6-a3cc-036baa3a5340 req-de0d60fd-1b8f-4397-b126-0c8dcc5deb93 service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] No waiting events found dispatching network-vif-plugged-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:00:01 user nova-compute[72081]: WARNING nova.compute.manager [req-995569ff-6838-4ac6-a3cc-036baa3a5340 req-de0d60fd-1b8f-4397-b126-0c8dcc5deb93 service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Received unexpected event network-vif-plugged-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda for instance with vm_state active and task_state rescuing.
Apr 12 04:00:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-c168d57b-1e4c-4de6-bea6-839c8fbd9cbe req-6faf4204-19e6-4438-86ad-0929ca9935f7 service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Received event network-vif-plugged-c83728df-a614-42d7-a164-d5d5e0373838 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:00:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c168d57b-1e4c-4de6-bea6-839c8fbd9cbe req-6faf4204-19e6-4438-86ad-0929ca9935f7 service nova] Acquiring lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:00:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c168d57b-1e4c-4de6-bea6-839c8fbd9cbe req-6faf4204-19e6-4438-86ad-0929ca9935f7 service nova] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:00:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c168d57b-1e4c-4de6-bea6-839c8fbd9cbe req-6faf4204-19e6-4438-86ad-0929ca9935f7 service nova] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:00:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-c168d57b-1e4c-4de6-bea6-839c8fbd9cbe req-6faf4204-19e6-4438-86ad-0929ca9935f7 service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] No waiting events found dispatching network-vif-plugged-c83728df-a614-42d7-a164-d5d5e0373838 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:00:03 user nova-compute[72081]: WARNING nova.compute.manager [req-c168d57b-1e4c-4de6-bea6-839c8fbd9cbe req-6faf4204-19e6-4438-86ad-0929ca9935f7 service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Received unexpected event network-vif-plugged-c83728df-a614-42d7-a164-d5d5e0373838 for instance with vm_state building and task_state spawning.
Apr 12 04:00:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-f50c2fd4-e9f5-4004-853b-2433c9452d11 req-cb43babd-2e99-4a5e-9763-d2ccfefbccd8 service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Received event network-vif-plugged-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:00:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f50c2fd4-e9f5-4004-853b-2433c9452d11 req-cb43babd-2e99-4a5e-9763-d2ccfefbccd8 service nova] Acquiring lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:00:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f50c2fd4-e9f5-4004-853b-2433c9452d11 req-cb43babd-2e99-4a5e-9763-d2ccfefbccd8 service nova] Lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:00:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f50c2fd4-e9f5-4004-853b-2433c9452d11 req-cb43babd-2e99-4a5e-9763-d2ccfefbccd8 service nova] Lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:00:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-f50c2fd4-e9f5-4004-853b-2433c9452d11 req-cb43babd-2e99-4a5e-9763-d2ccfefbccd8 service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] No waiting events found dispatching network-vif-plugged-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:00:03 user nova-compute[72081]: WARNING nova.compute.manager [req-f50c2fd4-e9f5-4004-853b-2433c9452d11 req-cb43babd-2e99-4a5e-9763-d2ccfefbccd8 service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Received unexpected event network-vif-plugged-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda for instance with vm_state active and task_state rescuing.
Apr 12 04:00:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-f50c2fd4-e9f5-4004-853b-2433c9452d11 req-cb43babd-2e99-4a5e-9763-d2ccfefbccd8 service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Received event network-vif-plugged-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:00:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f50c2fd4-e9f5-4004-853b-2433c9452d11 req-cb43babd-2e99-4a5e-9763-d2ccfefbccd8 service nova] Acquiring lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:00:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f50c2fd4-e9f5-4004-853b-2433c9452d11 req-cb43babd-2e99-4a5e-9763-d2ccfefbccd8 service nova] Lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:00:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f50c2fd4-e9f5-4004-853b-2433c9452d11 req-cb43babd-2e99-4a5e-9763-d2ccfefbccd8 service nova] Lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:00:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-f50c2fd4-e9f5-4004-853b-2433c9452d11 req-cb43babd-2e99-4a5e-9763-d2ccfefbccd8 service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] No waiting events found dispatching network-vif-plugged-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:00:03 user nova-compute[72081]: WARNING nova.compute.manager [req-f50c2fd4-e9f5-4004-853b-2433c9452d11 req-cb43babd-2e99-4a5e-9763-d2ccfefbccd8 service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Received unexpected event network-vif-plugged-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda for instance with vm_state active and task_state rescuing.
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.compute.manager [req-c6687a66-2ab9-4731-96e4-e6aaac1d3a17 req-9085985f-12a5-47e4-b41a-26e7a6c60fe2 service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Received event network-vif-plugged-c83728df-a614-42d7-a164-d5d5e0373838 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:00:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c6687a66-2ab9-4731-96e4-e6aaac1d3a17 req-9085985f-12a5-47e4-b41a-26e7a6c60fe2 service nova] Acquiring lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:00:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c6687a66-2ab9-4731-96e4-e6aaac1d3a17 req-9085985f-12a5-47e4-b41a-26e7a6c60fe2 service nova] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:00:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c6687a66-2ab9-4731-96e4-e6aaac1d3a17 req-9085985f-12a5-47e4-b41a-26e7a6c60fe2 service nova] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.compute.manager [req-c6687a66-2ab9-4731-96e4-e6aaac1d3a17 req-9085985f-12a5-47e4-b41a-26e7a6c60fe2 service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] No waiting events found dispatching network-vif-plugged-c83728df-a614-42d7-a164-d5d5e0373838 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:00:05 user nova-compute[72081]: WARNING nova.compute.manager [req-c6687a66-2ab9-4731-96e4-e6aaac1d3a17 req-9085985f-12a5-47e4-b41a-26e7a6c60fe2 service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Received unexpected event network-vif-plugged-c83728df-a614-42d7-a164-d5d5e0373838 for instance with vm_state building and task_state spawning.
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681272005.5311801, 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 04:00:05 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] VM Resumed (Lifecycle Event)
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.compute.manager [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 04:00:05 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Instance spawned successfully.
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.compute.manager [None req-d3f7d821-2946-49a0-93b1-d08640d5d5d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 04:00:05 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681272005.535352, 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 04:00:05 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] VM Started (Lifecycle Event)
Apr 12 04:00:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 04:00:05 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Removed pending event for 6cd92d70-a2af-4b2a-97c3-2981c157278d due to event {{(pid=72081) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681272005.5585818, 6cd92d70-a2af-4b2a-97c3-2981c157278d => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 04:00:05 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] VM Resumed (Lifecycle Event)
Apr 12 04:00:05 user nova-compute[72081]: INFO nova.compute.manager [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Took 7.90 seconds to spawn the instance on the hypervisor.
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.compute.manager [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: rescued, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681272005.5612767, 6cd92d70-a2af-4b2a-97c3-2981c157278d => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 04:00:05 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] VM Started (Lifecycle Event)
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 04:00:05 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Synchronizing instance power state after lifecycle event "Started"; current vm_state: rescued, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 04:00:05 user nova-compute[72081]: INFO nova.compute.manager [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Took 8.45 seconds to build instance.
Apr 12 04:00:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-2fa29020-cf11-45a7-a3ae-2df23c043582 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 8.544s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:00:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:00:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:00:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:00:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:00:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:22 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:00:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:26 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:00:27 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:00:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:00:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:00:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:00:27 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 04:00:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:00:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:00:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:00:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.141s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:00:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:00:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:00:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:00:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:00:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:00:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:00:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:00:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:00:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:00:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:00:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:00:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:00:29 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:00:29 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:00:29 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8519MB free_disk=25.946842193603516GB free_vcpus=9 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 04:00:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:00:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:00:29 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance de40e137-84e3-4d85-895b-1f6da2673457 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:00:29 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 6cd92d70-a2af-4b2a-97c3-2981c157278d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:00:29 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:00:29 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 04:00:29 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=896MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 04:00:29 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 04:00:29 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 04:00:29 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 04:00:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.241s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:00:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:32 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:00:32 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:00:32 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:00:32 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:00:32 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 04:00:33 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:00:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:35 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:00:36 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:00:36 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 04:00:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-6cd92d70-a2af-4b2a-97c3-2981c157278d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:00:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-6cd92d70-a2af-4b2a-97c3-2981c157278d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:00:36 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 04:00:37 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Updating instance_info_cache with network_info: [{"id": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "address": "fa:16:3e:c0:fc:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapee1cfaf9-15", "ovs_interfaceid": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:00:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-6cd92d70-a2af-4b2a-97c3-2981c157278d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:00:37 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 04:00:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:00:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:00:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:00:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:00:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:00:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:45 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:00:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:00:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:00:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:00:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:00:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:00:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:01:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:01:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:01:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:01:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:01:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:01:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:01:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:01:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:01:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:01:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:01:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:01:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:01:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4997-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:01:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:01:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:01:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:01:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:01:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:01:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:01:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:01:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:01:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:01:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:01:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:01:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:01:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:01:22 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:01:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:01:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:01:27 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:01:27 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:01:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:01:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:01:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:01:27 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 04:01:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:01:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.140s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:01:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:01:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.142s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:01:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:01:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:01:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:01:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:01:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:01:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:01:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:01:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:01:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:01:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:01:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:01:28 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:01:29 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:01:29 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:01:29 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8586MB free_disk=25.946151733398438GB free_vcpus=9 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 04:01:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:01:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:01:29 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance de40e137-84e3-4d85-895b-1f6da2673457 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:01:29 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 6cd92d70-a2af-4b2a-97c3-2981c157278d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:01:29 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:01:29 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 04:01:29 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=896MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 04:01:29 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Refreshing inventories for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 12 04:01:29 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Updating ProviderTree inventory for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 12 04:01:29 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Updating inventory in ProviderTree for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 12 04:01:29 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Refreshing aggregate associations for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6, aggregates: None {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 12 04:01:29 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Refreshing trait associations for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6, traits: COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_USB,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_RESCUE_BFV,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE41,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_STORAGE_BUS_FDC,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_STORAGE_BUS_IDE,COMPUTE_TRUSTED_CERTS,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_GRAPHICS_MODEL_CIRRUS,HW_CPU_X86_SSSE3,HW_CPU_X86_SSE,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_EXTEND,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_STORAGE_BUS_SATA,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_GRAPHICS_MODEL_QXL,HW_CPU_X86_SSE42,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_ACCELERATORS,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,HW_CPU_X86_MMX,COMPUTE_NODE,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_DEVICE_TAGGING,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_VIOMMU_MODEL_AUTO {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 12 04:01:29 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 04:01:29 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 04:01:29 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 04:01:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.473s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:01:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:01:33 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:01:33 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:01:33 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:01:33 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:01:33 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:01:33 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 04:01:33 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:01:34 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:01:34 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Cleaning up deleted instances with incomplete migration  {{(pid=72081) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 12 04:01:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:01:38 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:01:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 04:01:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Rebuilding the list of instances to heal {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 12 04:01:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:01:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:01:38 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 04:01:38 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lazy-loading 'info_cache' on Instance uuid de40e137-84e3-4d85-895b-1f6da2673457 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:01:39 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Updating instance_info_cache with network_info: [{"id": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "address": "fa:16:3e:c4:e2:d7", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap16a3f234-3d", "ovs_interfaceid": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:01:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:01:39 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 04:01:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:01:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:01:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:01:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:01:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:01:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:01:42 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:01:42 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Cleaning up deleted instances {{(pid=72081) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 12 04:01:42 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] There are 0 instances to clean {{(pid=72081) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 12 04:01:45 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:01:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:01:50 user nova-compute[72081]: DEBUG nova.compute.manager [req-c5157125-b3e5-436d-ac82-06bf77a9742c req-299b2c1a-c169-4e0e-a990-97f87db32c7d service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Received event network-changed-c83728df-a614-42d7-a164-d5d5e0373838 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:01:50 user nova-compute[72081]: DEBUG nova.compute.manager [req-c5157125-b3e5-436d-ac82-06bf77a9742c req-299b2c1a-c169-4e0e-a990-97f87db32c7d service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Refreshing instance network info cache due to event network-changed-c83728df-a614-42d7-a164-d5d5e0373838. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 04:01:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c5157125-b3e5-436d-ac82-06bf77a9742c req-299b2c1a-c169-4e0e-a990-97f87db32c7d service nova] Acquiring lock "refresh_cache-7a6fb26c-9c8f-4be7-b16f-8662eed2e489" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:01:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c5157125-b3e5-436d-ac82-06bf77a9742c req-299b2c1a-c169-4e0e-a990-97f87db32c7d service nova] Acquired lock "refresh_cache-7a6fb26c-9c8f-4be7-b16f-8662eed2e489" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:01:50 user nova-compute[72081]: DEBUG nova.network.neutron [req-c5157125-b3e5-436d-ac82-06bf77a9742c req-299b2c1a-c169-4e0e-a990-97f87db32c7d service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Refreshing network info cache for port c83728df-a614-42d7-a164-d5d5e0373838 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 04:01:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:01:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:01:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:01:50 user nova-compute[72081]: DEBUG nova.objects.instance [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lazy-loading 'flavor' on Instance uuid 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:01:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.063s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:01:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:01:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:01:50 user nova-compute[72081]: INFO nova.compute.manager [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Attaching volume 9c443292-90f6-4823-bfa7-fdec2393e3fe to /dev/vdb
Apr 12 04:01:51 user nova-compute[72081]: DEBUG nova.network.neutron [req-c5157125-b3e5-436d-ac82-06bf77a9742c req-299b2c1a-c169-4e0e-a990-97f87db32c7d service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Updated VIF entry in instance network info cache for port c83728df-a614-42d7-a164-d5d5e0373838. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 04:01:51 user nova-compute[72081]: DEBUG nova.network.neutron [req-c5157125-b3e5-436d-ac82-06bf77a9742c req-299b2c1a-c169-4e0e-a990-97f87db32c7d service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Updating instance_info_cache with network_info: [{"id": "c83728df-a614-42d7-a164-d5d5e0373838", "address": "fa:16:3e:b8:39:b0", "network": {"id": "1298d27c-9dfa-46e2-b941-024dd99ead79", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-1228939766-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.85", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01223543586e474b875e7dbf2c78ef87", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc83728df-a6", "ovs_interfaceid": "c83728df-a614-42d7-a164-d5d5e0373838", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:01:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c5157125-b3e5-436d-ac82-06bf77a9742c req-299b2c1a-c169-4e0e-a990-97f87db32c7d service nova] Releasing lock "refresh_cache-7a6fb26c-9c8f-4be7-b16f-8662eed2e489" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:01:51 user nova-compute[72081]: DEBUG os_brick.utils [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 04:01:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:01:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:01:51 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f78317a4-b44e-4188-8eb2-1ad9b67a53b9]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:01:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:01:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:01:51 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb046824-b9bc-4289-88f4-57a72a81beeb]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:01:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:01:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:01:51 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6d2cc36-fc7c-4c5a-a292-a52404e6daba]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:01:51 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e9ee153-2b97-4a5d-ad04-6c51da331b1c]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:01:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:01:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 04:01:51 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 04:01:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:01:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 04:01:51 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 04:01:51 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7aabae63-c640-4a78-ad72-7986c78bbcea]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:01:51 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 04:01:51 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 04:01:51 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 04:01:51 user nova-compute[72081]: DEBUG os_brick.utils [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] <== get_connector_properties: return (137ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 04:01:51 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Updating existing volume attachment record: 07080150-1d73-409f-b135-12fb0cf44558 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 04:01:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:01:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'jwREomNK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 04:01:56 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 04:01:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.003s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 04:01:56 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 04:01:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 04:01:56 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d41aab9c-b500-468b-8c3d-c6aa052ee5c4]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 04:01:56 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0070482b-a2a4-4a62-8dcb-47b3ce0141e8]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe] added
Apr 12 04:01:56 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[59623de3-e23a-4555-bdf0-fab932054f97]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[92328fc8-c27a-43df-ae00-dbe94724434b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe -p 172.16.0.220:3260 --op update -n node.session.auth.username -v jwREomNK {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe -p 172.16.0.220:3260 --op update -n node.session.auth.username -v jwREomNK" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ef86dcec-f7b5-463a-8393-7233a99f286f]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'jwREomNK'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.026s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9f29dc5d-ef20-44ce-b0c5-8b5849e112e0]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[de3b7a70-961f-4bf2-aa2f-d3e1198e9e75]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Apr 12 04:01:56 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Apr 12 04:01:56 user nova-compute[72081]:  {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 04:01:56 user nova-compute[72081]: WARNING os_brick.initiator.connectors.iscsi [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Apr 12 04:01:56 user nova-compute[72081]: 
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe -p 172.16.0.220:3260 --login" returned: 0 in 0.049s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[dee9eada-4f05-4b28-b14f-d23126d03ac5]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe, portal: 172.16.0.220,3260]
Apr 12 04:01:56 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe, portal: 172.16.0.220,3260] successful.
Apr 12 04:01:56 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[88a176f7-c919-45a3-97e3-99441e8a9961]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[834c70ec-b6ad-4e23-9203-87deb7c81770]: (4, ('tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe (non-flash)
Apr 12 04:01:56 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsi session list stdout=tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe (non-flash)
Apr 12 04:01:56 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.276s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] HCTL ('33', '-', '-', 0) found on session 50 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[66573577-8614-420c-bf4f-32bdf0f68c5c]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Searching for a device in session 50 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 04:01:56 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'jwREomNK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 04:01:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.308s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 04:01:57 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] <== connect_volume: return (1312ms) {'type': 'block', 'scsi_wwn': '23134353661663762', 'path': '/dev/sdb'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 04:01:57 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23134353661663762', 'path': '/dev/sdb'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 04:01:57 user nova-compute[72081]: DEBUG nova.objects.instance [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lazy-loading 'flavor' on Instance uuid 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:01:57 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] attach device xml: <disk type="block" device="disk">
Apr 12 04:01:57 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 04:01:57 user nova-compute[72081]:   <source dev="/dev/sdb"/>
Apr 12 04:01:57 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 04:01:57 user nova-compute[72081]:   <serial>9c443292-90f6-4823-bfa7-fdec2393e3fe</serial>
Apr 12 04:01:57 user nova-compute[72081]: </disk>
Apr 12 04:01:57 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 04:01:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 04:01:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 04:01:58 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] No VIF found with MAC fa:16:3e:b8:39:b0, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 04:01:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-1dced3f3-ebe9-4af2-b9de-ca0ffce9a0f0 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 7.571s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:01:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:02:05 user nova-compute[72081]: INFO nova.compute.manager [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Detaching volume 9c443292-90f6-4823-bfa7-fdec2393e3fe
Apr 12 04:02:05 user nova-compute[72081]: INFO nova.virt.block_device [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Attempting to driver detach volume 9c443292-90f6-4823-bfa7-fdec2393e3fe from mountpoint /dev/vdb
Apr 12 04:02:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Attempting to detach device vdb from instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 04:02:05 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 04:02:05 user nova-compute[72081]:   <source dev="/dev/sdb"/>
Apr 12 04:02:05 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 04:02:05 user nova-compute[72081]:   <serial>9c443292-90f6-4823-bfa7-fdec2393e3fe</serial>
Apr 12 04:02:05 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 04:02:05 user nova-compute[72081]: </disk>
Apr 12 04:02:05 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 04:02:05 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Successfully detached device vdb from instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 from the persistent domain config.
Apr 12 04:02:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 from the live domain config. {{(pid=72081) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 04:02:05 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 04:02:05 user nova-compute[72081]:   <source dev="/dev/sdb"/>
Apr 12 04:02:05 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 04:02:05 user nova-compute[72081]:   <serial>9c443292-90f6-4823-bfa7-fdec2393e3fe</serial>
Apr 12 04:02:05 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 04:02:05 user nova-compute[72081]: </disk>
Apr 12 04:02:05 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Received event <DeviceRemovedEvent: 1681272125.8169994, 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72081) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 {{(pid=72081) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 12 04:02:05 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Successfully detached device vdb from instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 from the live domain config.
Apr 12 04:02:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'jwREomNK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 04:02:05 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 04:02:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[92f22b6f-54c9-4087-86ea-94fb10641656]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 04:02:05 user nova-compute[72081]: No targets found.
Apr 12 04:02:05 user nova-compute[72081]: iSNS:
Apr 12 04:02:05 user nova-compute[72081]: No targets found.
Apr 12 04:02:05 user nova-compute[72081]: STATIC:
Apr 12 04:02:05 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe
Apr 12 04:02:05 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 04:02:05 user nova-compute[72081]:                 Iface Name: default
Apr 12 04:02:05 user nova-compute[72081]: FIRMWARE:
Apr 12 04:02:05 user nova-compute[72081]: No targets found.
Apr 12 04:02:05 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 04:02:05 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 04:02:05 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c002867c-c1d9-4275-8d65-82981377e0b8]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8e1a8d21-0ab1-4a23-bd1e-7a1e17208d5e]: (4, ('tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe (non-flash)
Apr 12 04:02:05 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsi session list stdout=tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe (non-flash)
Apr 12 04:02:05 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fac134ee0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe'): ({'sdb'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Removing single pathed devices sdb {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d7f64795-0024-4864-bfa4-f5d5f2bb8fa4]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[44a6f59e-bedd-43c5-90d4-4493732e7903]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Flushing IO for device /dev/sdb {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[63421fc1-436e-4000-bd09-650b74cff42d]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 04:02:05 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:02:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.044s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:02:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bf5fc4b6-11b3-4bf7-9eae-e0b8d2602eb5]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:02:06 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 04:02:06 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] SCSI volumes sdb have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 04:02:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 04:02:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:02:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:02:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[283a62dd-8c9b-43eb-9e0e-9e53ba3175a1]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:02:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:02:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:02:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe -p 172.16.0.220:3260 --logout" returned: 0 in 0.056s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:02:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[673d41d7-2aca-45c1-a939-3d26aa647e95]: (4, ('Logging out of session [sid: 50, target: iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe, portal: 172.16.0.220,3260]\nLogout of [sid: 50, target: iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:02:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 50, target: iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe, portal: 172.16.0.220,3260]
Apr 12 04:02:06 user nova-compute[72081]: Logout of [sid: 50, target: iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe, portal: 172.16.0.220,3260] successful.
Apr 12 04:02:06 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:02:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:02:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9c443292-90f6-4823-bfa7-fdec2393e3fe -p 172.16.0.220:3260 --op delete" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:02:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[616b3b88-8a07-45eb-8ce1-ab8f44980676]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:02:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:02:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.300s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 04:02:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] <== disconnect_volume: return (301ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 04:02:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 04:02:06 user nova-compute[72081]: DEBUG nova.objects.instance [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lazy-loading 'flavor' on Instance uuid 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:02:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f34abe42-537f-4d27-9cbd-208290c96fab tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.669s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:02:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:02:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:02:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:02:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:02:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:02:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:02:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:02:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:02:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:24 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:02:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:27 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:02:29 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:02:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:02:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:02:29 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:02:29 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 04:02:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:02:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:02:29 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:02:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.142s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:02:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:02:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:02:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:02:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:02:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:02:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:02:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:02:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:02:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:02:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:02:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:02:30 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:02:31 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:02:31 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:02:31 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8557MB free_disk=25.944866180419922GB free_vcpus=9 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 04:02:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:02:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:02:31 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance de40e137-84e3-4d85-895b-1f6da2673457 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:02:31 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 6cd92d70-a2af-4b2a-97c3-2981c157278d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:02:31 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:02:31 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 04:02:31 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=896MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 04:02:31 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 04:02:31 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 04:02:31 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 04:02:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.228s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:02:34 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:02:34 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:02:34 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:02:34 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:02:34 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:02:34 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 04:02:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:37 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:02:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:40 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:02:40 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 04:02:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-6cd92d70-a2af-4b2a-97c3-2981c157278d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:02:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-6cd92d70-a2af-4b2a-97c3-2981c157278d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:02:40 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 04:02:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:41 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Updating instance_info_cache with network_info: [{"id": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "address": "fa:16:3e:c0:fc:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapee1cfaf9-15", "ovs_interfaceid": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:02:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-6cd92d70-a2af-4b2a-97c3-2981c157278d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:02:41 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 04:02:45 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:02:45 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:45 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:02:45 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:02:45 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:02:45 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:50 user nova-compute[72081]: INFO nova.compute.manager [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Rescuing
Apr 12 04:02:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:02:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquired lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:02:50 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 04:02:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:02:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:02:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:02:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:02:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:02:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:50 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Updating instance_info_cache with network_info: [{"id": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "address": "fa:16:3e:c4:e2:d7", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap16a3f234-3d", "ovs_interfaceid": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:02:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Releasing lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:02:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.compute.manager [req-f21b6864-b687-45ad-ab06-5342b8313f9b req-ddd3665e-bb51-434b-b4a4-461750f9c201 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received event network-vif-unplugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f21b6864-b687-45ad-ab06-5342b8313f9b req-ddd3665e-bb51-434b-b4a4-461750f9c201 service nova] Acquiring lock "de40e137-84e3-4d85-895b-1f6da2673457-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f21b6864-b687-45ad-ab06-5342b8313f9b req-ddd3665e-bb51-434b-b4a4-461750f9c201 service nova] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f21b6864-b687-45ad-ab06-5342b8313f9b req-ddd3665e-bb51-434b-b4a4-461750f9c201 service nova] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.compute.manager [req-f21b6864-b687-45ad-ab06-5342b8313f9b req-ddd3665e-bb51-434b-b4a4-461750f9c201 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] No waiting events found dispatching network-vif-unplugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:02:51 user nova-compute[72081]: WARNING nova.compute.manager [req-f21b6864-b687-45ad-ab06-5342b8313f9b req-ddd3665e-bb51-434b-b4a4-461750f9c201 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received unexpected event network-vif-unplugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c for instance with vm_state active and task_state rescuing.
Apr 12 04:02:51 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:51 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Instance destroyed successfully.
Apr 12 04:02:51 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Attempting rescue
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} {{(pid=72081) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4289}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Instance directory exists: not creating {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4694}}
Apr 12 04:02:51 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Creating image(s)
Apr 12 04:02:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "/opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "/opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "/opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.007s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'trusted_certs' on Instance uuid de40e137-84e3-4d85-895b-1f6da2673457 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.130s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk.rescue {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk.rescue" returned: 0 in 0.051s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.188s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'migration_context' on Instance uuid de40e137-84e3-4d85-895b-1f6da2673457 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Start _get_guest_xml network_info=[{"id": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "address": "fa:16:3e:c4:e2:d7", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "vif_mac": "fa:16:3e:c4:e2:d7"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap16a3f234-3d", "ovs_interfaceid": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7', 'kernel_id': '', 'ramdisk_id': ''} block_device_info=None {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'resources' on Instance uuid de40e137-84e3-4d85-895b-1f6da2673457 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'numa_topology' on Instance uuid de40e137-84e3-4d85-895b-1f6da2673457 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:02:51 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:02:51 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'vcpu_model' on Instance uuid de40e137-84e3-4d85-895b-1f6da2673457 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:58:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-1232389953',display_name='tempest-ServerRescueNegativeTestJSON-server-1232389953',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-1232389953',id=44,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-12T03:58:14Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='a6a8a4f5d59e479f9ca7b73a9e270c4b',ramdisk_id='',reservation_id='r-tsy8s5du',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-529226261',owner_user_name='tempest-ServerRescueNegativeTestJSON-529226261-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T03:58:15Z,user_data=None,user_id='44a0206f1323440db4f8411a3c3c088d',uuid=de40e137-84e3-4d85-895b-1f6da2673457,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "address": "fa:16:3e:c4:e2:d7", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "vif_mac": "fa:16:3e:c4:e2:d7"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap16a3f234-3d", "ovs_interfaceid": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Converting VIF {"id": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "address": "fa:16:3e:c4:e2:d7", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "vif_mac": "fa:16:3e:c4:e2:d7"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap16a3f234-3d", "ovs_interfaceid": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:c4:e2:d7,bridge_name='br-int',has_traffic_filtering=True,id=16a3f234-3d0d-446c-b6a0-0a240e35aa6c,network=Network(311bda05-075c-4bb3-9469-b5117dd33353),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap16a3f234-3d') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 04:02:51 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'pci_devices' on Instance uuid de40e137-84e3-4d85-895b-1f6da2673457 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:02:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] End _get_guest_xml xml=<domain type="kvm">
Apr 12 04:02:52 user nova-compute[72081]:   <uuid>de40e137-84e3-4d85-895b-1f6da2673457</uuid>
Apr 12 04:02:52 user nova-compute[72081]:   <name>instance-0000002c</name>
Apr 12 04:02:52 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 04:02:52 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 04:02:52 user nova-compute[72081]:   <metadata>
Apr 12 04:02:52 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 04:02:52 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 04:02:52 user nova-compute[72081]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-1232389953</nova:name>
Apr 12 04:02:52 user nova-compute[72081]:       <nova:creationTime>2023-04-12 04:02:51</nova:creationTime>
Apr 12 04:02:52 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 04:02:52 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 04:02:52 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 04:02:52 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 04:02:52 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 04:02:52 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 04:02:52 user nova-compute[72081]:       </nova:flavor>
Apr 12 04:02:52 user nova-compute[72081]:       <nova:owner>
Apr 12 04:02:52 user nova-compute[72081]:         <nova:user uuid="44a0206f1323440db4f8411a3c3c088d">tempest-ServerRescueNegativeTestJSON-529226261-project-member</nova:user>
Apr 12 04:02:52 user nova-compute[72081]:         <nova:project uuid="a6a8a4f5d59e479f9ca7b73a9e270c4b">tempest-ServerRescueNegativeTestJSON-529226261</nova:project>
Apr 12 04:02:52 user nova-compute[72081]:       </nova:owner>
Apr 12 04:02:52 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 04:02:52 user nova-compute[72081]:       <nova:ports>
Apr 12 04:02:52 user nova-compute[72081]:         <nova:port uuid="16a3f234-3d0d-446c-b6a0-0a240e35aa6c">
Apr 12 04:02:52 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Apr 12 04:02:52 user nova-compute[72081]:         </nova:port>
Apr 12 04:02:52 user nova-compute[72081]:       </nova:ports>
Apr 12 04:02:52 user nova-compute[72081]:     </nova:instance>
Apr 12 04:02:52 user nova-compute[72081]:   </metadata>
Apr 12 04:02:52 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 04:02:52 user nova-compute[72081]:     <system>
Apr 12 04:02:52 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 04:02:52 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 04:02:52 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 04:02:52 user nova-compute[72081]:       <entry name="serial">de40e137-84e3-4d85-895b-1f6da2673457</entry>
Apr 12 04:02:52 user nova-compute[72081]:       <entry name="uuid">de40e137-84e3-4d85-895b-1f6da2673457</entry>
Apr 12 04:02:52 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 04:02:52 user nova-compute[72081]:     </system>
Apr 12 04:02:52 user nova-compute[72081]:   </sysinfo>
Apr 12 04:02:52 user nova-compute[72081]:   <os>
Apr 12 04:02:52 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 04:02:52 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 04:02:52 user nova-compute[72081]:   </os>
Apr 12 04:02:52 user nova-compute[72081]:   <features>
Apr 12 04:02:52 user nova-compute[72081]:     <acpi/>
Apr 12 04:02:52 user nova-compute[72081]:     <apic/>
Apr 12 04:02:52 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 04:02:52 user nova-compute[72081]:   </features>
Apr 12 04:02:52 user nova-compute[72081]:   <clock offset="utc">
Apr 12 04:02:52 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 04:02:52 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 04:02:52 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 04:02:52 user nova-compute[72081]:   </clock>
Apr 12 04:02:52 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 04:02:52 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 04:02:52 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 04:02:52 user nova-compute[72081]:   </cpu>
Apr 12 04:02:52 user nova-compute[72081]:   <devices>
Apr 12 04:02:52 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 04:02:52 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 04:02:52 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk.rescue"/>
Apr 12 04:02:52 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 04:02:52 user nova-compute[72081]:     </disk>
Apr 12 04:02:52 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 04:02:52 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 04:02:52 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk"/>
Apr 12 04:02:52 user nova-compute[72081]:       <target dev="vdb" bus="virtio"/>
Apr 12 04:02:52 user nova-compute[72081]:     </disk>
Apr 12 04:02:52 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 04:02:52 user nova-compute[72081]:       <mac address="fa:16:3e:c4:e2:d7"/>
Apr 12 04:02:52 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 04:02:52 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 04:02:52 user nova-compute[72081]:       <target dev="tap16a3f234-3d"/>
Apr 12 04:02:52 user nova-compute[72081]:     </interface>
Apr 12 04:02:52 user nova-compute[72081]:     <serial type="pty">
Apr 12 04:02:52 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/console.log" append="off"/>
Apr 12 04:02:52 user nova-compute[72081]:     </serial>
Apr 12 04:02:52 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 04:02:52 user nova-compute[72081]:     <video>
Apr 12 04:02:52 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 04:02:52 user nova-compute[72081]:     </video>
Apr 12 04:02:52 user nova-compute[72081]:     <rng model="virtio">
Apr 12 04:02:52 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 04:02:52 user nova-compute[72081]:     </rng>
Apr 12 04:02:52 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 04:02:52 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 04:02:52 user nova-compute[72081]:       <stats period="10"/>
Apr 12 04:02:52 user nova-compute[72081]:     </memballoon>
Apr 12 04:02:52 user nova-compute[72081]:   </devices>
Apr 12 04:02:52 user nova-compute[72081]: </domain>
Apr 12 04:02:52 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 04:02:52 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Instance destroyed successfully.
Apr 12 04:02:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 04:02:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 04:02:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] No VIF found with MAC fa:16:3e:c4:e2:d7, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 04:02:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-98fbb7db-917b-4999-a087-df5e777fba91 req-8cb889fe-59c8-495d-a617-159685289815 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received event network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:02:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-98fbb7db-917b-4999-a087-df5e777fba91 req-8cb889fe-59c8-495d-a617-159685289815 service nova] Acquiring lock "de40e137-84e3-4d85-895b-1f6da2673457-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:02:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-98fbb7db-917b-4999-a087-df5e777fba91 req-8cb889fe-59c8-495d-a617-159685289815 service nova] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:02:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-98fbb7db-917b-4999-a087-df5e777fba91 req-8cb889fe-59c8-495d-a617-159685289815 service nova] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:02:53 user nova-compute[72081]: DEBUG nova.compute.manager [req-98fbb7db-917b-4999-a087-df5e777fba91 req-8cb889fe-59c8-495d-a617-159685289815 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] No waiting events found dispatching network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:02:53 user nova-compute[72081]: WARNING nova.compute.manager [req-98fbb7db-917b-4999-a087-df5e777fba91 req-8cb889fe-59c8-495d-a617-159685289815 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received unexpected event network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c for instance with vm_state active and task_state rescuing.
Apr 12 04:02:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:55 user nova-compute[72081]: DEBUG nova.compute.manager [req-5c3f944b-5179-457d-b2c2-330318e4d943 req-0783cdfd-937b-4ca9-86fd-42510c22936c service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received event network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:02:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5c3f944b-5179-457d-b2c2-330318e4d943 req-0783cdfd-937b-4ca9-86fd-42510c22936c service nova] Acquiring lock "de40e137-84e3-4d85-895b-1f6da2673457-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:02:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5c3f944b-5179-457d-b2c2-330318e4d943 req-0783cdfd-937b-4ca9-86fd-42510c22936c service nova] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:02:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5c3f944b-5179-457d-b2c2-330318e4d943 req-0783cdfd-937b-4ca9-86fd-42510c22936c service nova] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:02:55 user nova-compute[72081]: DEBUG nova.compute.manager [req-5c3f944b-5179-457d-b2c2-330318e4d943 req-0783cdfd-937b-4ca9-86fd-42510c22936c service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] No waiting events found dispatching network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:02:55 user nova-compute[72081]: WARNING nova.compute.manager [req-5c3f944b-5179-457d-b2c2-330318e4d943 req-0783cdfd-937b-4ca9-86fd-42510c22936c service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received unexpected event network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c for instance with vm_state active and task_state rescuing.
Apr 12 04:02:55 user nova-compute[72081]: DEBUG nova.compute.manager [req-5c3f944b-5179-457d-b2c2-330318e4d943 req-0783cdfd-937b-4ca9-86fd-42510c22936c service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received event network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:02:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5c3f944b-5179-457d-b2c2-330318e4d943 req-0783cdfd-937b-4ca9-86fd-42510c22936c service nova] Acquiring lock "de40e137-84e3-4d85-895b-1f6da2673457-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:02:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5c3f944b-5179-457d-b2c2-330318e4d943 req-0783cdfd-937b-4ca9-86fd-42510c22936c service nova] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:02:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5c3f944b-5179-457d-b2c2-330318e4d943 req-0783cdfd-937b-4ca9-86fd-42510c22936c service nova] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:02:55 user nova-compute[72081]: DEBUG nova.compute.manager [req-5c3f944b-5179-457d-b2c2-330318e4d943 req-0783cdfd-937b-4ca9-86fd-42510c22936c service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] No waiting events found dispatching network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:02:55 user nova-compute[72081]: WARNING nova.compute.manager [req-5c3f944b-5179-457d-b2c2-330318e4d943 req-0783cdfd-937b-4ca9-86fd-42510c22936c service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received unexpected event network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c for instance with vm_state active and task_state rescuing.
Apr 12 04:02:55 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Removed pending event for de40e137-84e3-4d85-895b-1f6da2673457 due to event {{(pid=72081) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 12 04:02:55 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681272175.487857, de40e137-84e3-4d85-895b-1f6da2673457 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 04:02:55 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] VM Resumed (Lifecycle Event)
Apr 12 04:02:55 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4f471705-eefc-426a-81b4-fe3a47ad8408 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 04:02:55 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 04:02:55 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 04:02:55 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] During sync_power_state the instance has a pending task (rescuing). Skip.
Apr 12 04:02:55 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681272175.4888678, de40e137-84e3-4d85-895b-1f6da2673457 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 04:02:55 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] VM Started (Lifecycle Event)
Apr 12 04:02:55 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 04:02:55 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 04:02:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG nova.objects.instance [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lazy-loading 'flavor' on Instance uuid 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.051s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:02:58 user nova-compute[72081]: INFO nova.compute.manager [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Attaching volume c4b84f00-ee57-457a-924e-f2d7f2aada61 to /dev/vdb
Apr 12 04:02:58 user nova-compute[72081]: DEBUG os_brick.utils [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[51c95468-7062-4bb5-9060-f9e2302d834c]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9e83eeed-6675-438e-aa13-9daae7361583]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f608d94-f563-4876-ad75-a65ba157f72f]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[14e9d366-31ff-4d23-8bb9-f62a690d450d]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 04:02:58 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 04:02:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5ad94f2-b315-4eaa-b0cc-7abefe278636]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG os_brick.utils [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] <== get_connector_properties: return (122ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Updating existing volume attachment record: 587ba784-8608-4a8c-96eb-2e034d815a57 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 04:02:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:03:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'SQfcGNmz', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 04:03:02 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 04:03:02 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 04:03:02 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 04:03:02 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 04:03:02 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7fc5cc50-704a-4826-93e4-5b5a57562d09]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 04:03:02 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[31532765-98c1-4fc5-9667-2d306ce68658]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61] added
Apr 12 04:03:02 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[68bbf80c-91f1-49fc-88b7-4a4e9ca1043a]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d0a32a36-afd3-4503-a135-f73e357ba96b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v SQfcGNmz {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v SQfcGNmz" returned: 0 in 0.058s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c0fe722a-c553-4b4f-ae6d-ca539007d781]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'SQfcGNmz'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a69d9878-27fd-4c5d-a09c-491cfe821db7]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[12e268e7-c8c5-4483-9413-78f124aae693]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Apr 12 04:03:02 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Apr 12 04:03:02 user nova-compute[72081]:  {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 04:03:02 user nova-compute[72081]: WARNING os_brick.initiator.connectors.iscsi [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Apr 12 04:03:02 user nova-compute[72081]: 
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 -p 172.16.0.220:3260 --login" returned: 0 in 0.045s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6ee09978-994c-44d9-bab0-d71c48cbb5c5]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61, portal: 172.16.0.220,3260]
Apr 12 04:03:02 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61, portal: 172.16.0.220,3260] successful.
Apr 12 04:03:02 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[80661201-8017-47a3-8de9-8530da21d6b9]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:03:02 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ab3a7af2-0e69-4a4a-bb05-38af6f304e1b]: (4, ('tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 (non-flash)
Apr 12 04:03:03 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 04:03:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsi session list stdout=tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 (non-flash)
Apr 12 04:03:03 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 04:03:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.300s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 04:03:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 04:03:03 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] HCTL ('33', '-', '-', 0) found on session 51 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 04:03:03 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 04:03:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f8ce6fc-bf81-401c-be54-4567e92133c7]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:03 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Searching for a device in session 51 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 04:03:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'SQfcGNmz', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 04:03:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:03:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.331s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 04:03:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] <== connect_volume: return (1332ms) {'type': 'block', 'scsi_wwn': '23032656138326137', 'path': '/dev/sdb'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 04:03:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23032656138326137', 'path': '/dev/sdb'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 04:03:04 user nova-compute[72081]: DEBUG nova.objects.instance [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lazy-loading 'flavor' on Instance uuid 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:03:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] attach device xml: <disk type="block" device="disk">
Apr 12 04:03:04 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 04:03:04 user nova-compute[72081]:   <source dev="/dev/sdb"/>
Apr 12 04:03:04 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 04:03:04 user nova-compute[72081]:   <serial>c4b84f00-ee57-457a-924e-f2d7f2aada61</serial>
Apr 12 04:03:04 user nova-compute[72081]: </disk>
Apr 12 04:03:04 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 04:03:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 04:03:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 04:03:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] No VIF found with MAC fa:16:3e:b8:39:b0, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 04:03:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-69ea65f4-ad02-4b17-8433-b0b3c874a990 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.242s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:03:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:03:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:03:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:03:05 user nova-compute[72081]: INFO nova.compute.manager [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Detaching volume c4b84f00-ee57-457a-924e-f2d7f2aada61
Apr 12 04:03:05 user nova-compute[72081]: INFO nova.virt.block_device [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Attempting to driver detach volume c4b84f00-ee57-457a-924e-f2d7f2aada61 from mountpoint /dev/vdb
Apr 12 04:03:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Attempting to detach device vdb from instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 04:03:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 04:03:05 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 04:03:05 user nova-compute[72081]:   <source dev="/dev/sdb"/>
Apr 12 04:03:05 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 04:03:05 user nova-compute[72081]:   <serial>c4b84f00-ee57-457a-924e-f2d7f2aada61</serial>
Apr 12 04:03:05 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 04:03:05 user nova-compute[72081]: </disk>
Apr 12 04:03:05 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 04:03:05 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Successfully detached device vdb from instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 from the persistent domain config.
Apr 12 04:03:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 from the live domain config. {{(pid=72081) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 12 04:03:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 04:03:05 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 04:03:05 user nova-compute[72081]:   <source dev="/dev/sdb"/>
Apr 12 04:03:05 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 04:03:05 user nova-compute[72081]:   <serial>c4b84f00-ee57-457a-924e-f2d7f2aada61</serial>
Apr 12 04:03:05 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 04:03:05 user nova-compute[72081]: </disk>
Apr 12 04:03:05 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Received event <DeviceRemovedEvent: 1681272186.3851464, 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72081) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 {{(pid=72081) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 12 04:03:06 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Successfully detached device vdb from instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 from the live domain config.
Apr 12 04:03:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'SQfcGNmz', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 04:03:06 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 04:03:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e99b370a-7df2-48cd-bd8f-6d8f948eb644]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 04:03:06 user nova-compute[72081]: No targets found.
Apr 12 04:03:06 user nova-compute[72081]: iSNS:
Apr 12 04:03:06 user nova-compute[72081]: No targets found.
Apr 12 04:03:06 user nova-compute[72081]: STATIC:
Apr 12 04:03:06 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61
Apr 12 04:03:06 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 04:03:06 user nova-compute[72081]:                 Iface Name: default
Apr 12 04:03:06 user nova-compute[72081]: FIRMWARE:
Apr 12 04:03:06 user nova-compute[72081]: No targets found.
Apr 12 04:03:06 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 04:03:06 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 04:03:06 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bce30076-13b1-4f34-8b41-426a34806bf0]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6e0ad6ef-3b0c-4cb9-abd1-433b1824ec17]: (4, ('tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 (non-flash)
Apr 12 04:03:06 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsi session list stdout=tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 (non-flash)
Apr 12 04:03:06 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3fb420edd0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61'): ({'sdb'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Removing single pathed devices sdb {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ea82c0b0-0864-4a9b-b896-2032c156186b]: (4, ('path checker states:\npending             1\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d64e3ceb-41bf-449c-8f92-5627a4ee8e5f]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Flushing IO for device /dev/sdb {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.013s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[57d02803-a347-45e3-a001-cc06f0b92926]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.045s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb9b87e3-8587-40eb-998f-d4098853422b]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] SCSI volumes sdb have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.055s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bec2de39-6246-4e34-8b0f-347dd1ac87d9]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 -p 172.16.0.220:3260 --logout" returned: 0 in 0.121s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3e62c84d-e597-4604-9540-4e1d5630c87a]: (4, ('Logging out of session [sid: 51, target: iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61, portal: 172.16.0.220,3260]\nLogout of [sid: 51, target: iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 51, target: iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61, portal: 172.16.0.220,3260]
Apr 12 04:03:06 user nova-compute[72081]: Logout of [sid: 51, target: iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61, portal: 172.16.0.220,3260] successful.
Apr 12 04:03:06 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c4b84f00-ee57-457a-924e-f2d7f2aada61 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[10963370-303c-4ef6-91d5-871929162492]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.387s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] <== disconnect_volume: return (388ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 04:03:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 04:03:07 user nova-compute[72081]: DEBUG nova.objects.instance [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lazy-loading 'flavor' on Instance uuid 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:03:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-f86fba55-abb3-4211-9e85-f48f73ca9b05 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.921s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:03:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:03:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:03:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:03:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:03:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:03:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:03:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:03:25 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:03:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:03:27 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:03:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:03:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:03:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:03:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:03:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:03:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:03:31 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:03:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:03:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:03:31 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:03:31 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 04:03:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk.rescue --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk.rescue --force-share --output=json" returned: 0 in 0.132s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:33 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:03:33 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:03:33 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8562MB free_disk=25.924152374267578GB free_vcpus=9 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 04:03:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:03:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:03:33 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance de40e137-84e3-4d85-895b-1f6da2673457 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:03:33 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 6cd92d70-a2af-4b2a-97c3-2981c157278d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:03:33 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:03:33 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 04:03:33 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=896MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 04:03:33 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 04:03:33 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 04:03:33 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 04:03:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.247s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:03:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:03:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:03:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:03:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:03:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:03:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:03:35 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:03:35 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:03:35 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:03:35 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:03:35 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:03:35 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 04:03:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:03:42 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:03:42 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 04:03:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-7a6fb26c-9c8f-4be7-b16f-8662eed2e489" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:03:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-7a6fb26c-9c8f-4be7-b16f-8662eed2e489" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:03:42 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 04:03:43 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Updating instance_info_cache with network_info: [{"id": "c83728df-a614-42d7-a164-d5d5e0373838", "address": "fa:16:3e:b8:39:b0", "network": {"id": "1298d27c-9dfa-46e2-b941-024dd99ead79", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-1228939766-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.85", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01223543586e474b875e7dbf2c78ef87", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc83728df-a6", "ovs_interfaceid": "c83728df-a614-42d7-a164-d5d5e0373838", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:03:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-7a6fb26c-9c8f-4be7-b16f-8662eed2e489" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:03:43 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 04:03:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:03:45 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:03:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:03:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:03:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:03:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:03:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:03:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:03:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:03:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:03:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:03:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:03:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:03:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:03:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:03:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:03:58 user nova-compute[72081]: DEBUG nova.objects.instance [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lazy-loading 'flavor' on Instance uuid 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:03:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.050s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:03:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:03:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:03:58 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:03:58 user nova-compute[72081]: INFO nova.compute.manager [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Attaching volume 74d431c2-3cc6-458a-bc7b-c267be57fb16 to /dev/vdb
Apr 12 04:03:58 user nova-compute[72081]: DEBUG os_brick.utils [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 04:03:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f2f561dd-53a9-4605-a13f-a86972da0a2e]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a6e0d1f1-e7ae-450a-a5a4-9e16c496bd22]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:03:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[16114b07-93d5-44f3-b5b1-264f527f79cf]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ffe21a7e-f45e-4bc7-a8ae-33b4c75c7c50]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 04:03:58 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 04:03:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:03:58 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 04:03:58 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 04:03:58 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c56455fb-f47f-45af-a320-9f2b4d59d342]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:03:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 04:03:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 04:03:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 04:03:59 user nova-compute[72081]: DEBUG os_brick.utils [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] <== get_connector_properties: return (136ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 04:03:59 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Updating existing volume attachment record: 9060beac-f22a-4bdf-9f9a-4afd318a86a3 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 04:04:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'rVqTKHvY', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 04:04:03 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 04:04:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 04:04:03 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 04:04:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 04:04:03 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4759e7d8-d5e7-4da2-973c-09110c38d2d5]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 04:04:03 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b7eaee4c-7dc6-4991-996f-c9d2f1364eda]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16] added
Apr 12 04:04:03 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e7271a71-04a1-4b31-888d-e8b45aa121f2]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4c0c5058-5e52-48b5-8d59-9aa18b40d26c]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v rVqTKHvY {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v rVqTKHvY" returned: 0 in 0.028s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8e2bf136-6f55-4540-abba-1c7b31c669ff]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'rVqTKHvY'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.023s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[141f774e-9127-49d0-98da-587a40e50e55]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b722d10e-753e-444a-aaaa-7291ebd70c98]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Apr 12 04:04:03 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Apr 12 04:04:03 user nova-compute[72081]:  {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 04:04:03 user nova-compute[72081]: WARNING os_brick.initiator.connectors.iscsi [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Apr 12 04:04:03 user nova-compute[72081]: 
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 -p 172.16.0.220:3260 --login" returned: 0 in 0.048s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4e4132aa-da0b-49b1-a3ab-d16574732c1e]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16, portal: 172.16.0.220,3260]
Apr 12 04:04:03 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16, portal: 172.16.0.220,3260] successful.
Apr 12 04:04:03 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b412f5d1-2329-4b82-b65c-506363f91fe3]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4c701bc1-1597-4d52-8499-33a0b3212712]: (4, ('tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 (non-flash)
Apr 12 04:04:03 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsi session list stdout=tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 (non-flash)
Apr 12 04:04:03 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.275s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] HCTL ('33', '-', '-', 0) found on session 52 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[95f35141-15d2-46ba-841e-14ae45156619]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Searching for a device in session 52 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'rVqTKHvY', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 04:04:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.304s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 04:04:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] <== connect_volume: return (1307ms) {'type': 'block', 'scsi_wwn': '23538616432623161', 'path': '/dev/sdb'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 04:04:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23538616432623161', 'path': '/dev/sdb'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 04:04:04 user nova-compute[72081]: DEBUG nova.objects.instance [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lazy-loading 'flavor' on Instance uuid 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:04:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] attach device xml: <disk type="block" device="disk">
Apr 12 04:04:04 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 04:04:04 user nova-compute[72081]:   <source dev="/dev/sdb"/>
Apr 12 04:04:04 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 04:04:04 user nova-compute[72081]:   <serial>74d431c2-3cc6-458a-bc7b-c267be57fb16</serial>
Apr 12 04:04:04 user nova-compute[72081]: </disk>
Apr 12 04:04:04 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 04:04:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 04:04:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 04:04:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] No VIF found with MAC fa:16:3e:b8:39:b0, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 04:04:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-c6ab5b78-525a-4882-923c-0d96fcc2401e tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.228s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:04:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:04:06 user nova-compute[72081]: INFO nova.compute.manager [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Detaching volume 74d431c2-3cc6-458a-bc7b-c267be57fb16
Apr 12 04:04:06 user nova-compute[72081]: INFO nova.virt.block_device [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Attempting to driver detach volume 74d431c2-3cc6-458a-bc7b-c267be57fb16 from mountpoint /dev/vdb
Apr 12 04:04:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Attempting to detach device vdb from instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 04:04:06 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 04:04:06 user nova-compute[72081]:   <source dev="/dev/sdb"/>
Apr 12 04:04:06 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 04:04:06 user nova-compute[72081]:   <serial>74d431c2-3cc6-458a-bc7b-c267be57fb16</serial>
Apr 12 04:04:06 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 04:04:06 user nova-compute[72081]: </disk>
Apr 12 04:04:06 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 04:04:06 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Successfully detached device vdb from instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 from the persistent domain config.
Apr 12 04:04:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 from the live domain config. {{(pid=72081) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 04:04:06 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 04:04:06 user nova-compute[72081]:   <source dev="/dev/sdb"/>
Apr 12 04:04:06 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 04:04:06 user nova-compute[72081]:   <serial>74d431c2-3cc6-458a-bc7b-c267be57fb16</serial>
Apr 12 04:04:06 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 04:04:06 user nova-compute[72081]: </disk>
Apr 12 04:04:06 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Received event <DeviceRemovedEvent: 1681272246.6473744, 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72081) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 {{(pid=72081) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 12 04:04:06 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Successfully detached device vdb from instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 from the live domain config.
Apr 12 04:04:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'rVqTKHvY', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 04:04:06 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 04:04:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7866b282-49c7-4466-996f-ad35790b2a92]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 04:04:06 user nova-compute[72081]: No targets found.
Apr 12 04:04:06 user nova-compute[72081]: iSNS:
Apr 12 04:04:06 user nova-compute[72081]: No targets found.
Apr 12 04:04:06 user nova-compute[72081]: STATIC:
Apr 12 04:04:06 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16
Apr 12 04:04:06 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 04:04:06 user nova-compute[72081]:                 Iface Name: default
Apr 12 04:04:06 user nova-compute[72081]: FIRMWARE:
Apr 12 04:04:06 user nova-compute[72081]: No targets found.
Apr 12 04:04:06 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 04:04:06 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 04:04:06 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[879193d3-cc71-43d8-b5e5-657ac8fc2cc4]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0df289a4-4dd6-4b9f-8d8b-d3fc8f29bbcd]: (4, ('tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 (non-flash)
Apr 12 04:04:06 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsi session list stdout=tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 (non-flash)
Apr 12 04:04:06 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3f9416d630>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16'): ({'sdb'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Removing single pathed devices sdb {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2bc887d6-f112-41c0-aeb6-63c5a105d63a]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[41e23edf-36a0-4330-943b-ddf6b5a5797b]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Flushing IO for device /dev/sdb {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5dfe6d65-1568-4e0f-b166-74af99e01551]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.045s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9dfbefd0-c5b6-4c53-ba86-5d4ed9c53eba]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] SCSI volumes sdb have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a6b44c7e-4f57-4b28-9ced-4cf9c68c4058]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 -p 172.16.0.220:3260 --logout" returned: 0 in 0.048s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7caa59a3-3763-4fad-b294-26df04c10558]: (4, ('Logging out of session [sid: 52, target: iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16, portal: 172.16.0.220,3260]\nLogout of [sid: 52, target: iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 52, target: iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16, portal: 172.16.0.220,3260]
Apr 12 04:04:06 user nova-compute[72081]: Logout of [sid: 52, target: iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16, portal: 172.16.0.220,3260] successful.
Apr 12 04:04:06 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:74d431c2-3cc6-458a-bc7b-c267be57fb16 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b04318ea-42c3-46bd-a586-5252a553d2e4]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.276s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] <== disconnect_volume: return (277ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 04:04:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 04:04:07 user nova-compute[72081]: DEBUG nova.objects.instance [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lazy-loading 'flavor' on Instance uuid 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:04:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d84ab057-248c-4646-b027-81da130f4a20 tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.642s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:04:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:04:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:25 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:04:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:28 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:04:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:04:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:04:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:04:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:04:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:32 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:04:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:04:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:04:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:04:32 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 04:04:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.182s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.142s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk.rescue --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk.rescue --force-share --output=json" returned: 0 in 0.152s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:34 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:04:34 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:04:34 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8578MB free_disk=25.923107147216797GB free_vcpus=9 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 04:04:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:04:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:04:34 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance de40e137-84e3-4d85-895b-1f6da2673457 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:04:34 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 6cd92d70-a2af-4b2a-97c3-2981c157278d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:04:34 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:04:34 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 04:04:34 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=896MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 04:04:34 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 04:04:34 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 04:04:34 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 04:04:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.239s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:04:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:36 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:04:36 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:04:36 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:04:36 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:04:36 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 04:04:37 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:04:40 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:04:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:04:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:04:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:04:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:04:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:04:41 user nova-compute[72081]: INFO nova.compute.manager [None req-4f2f11fd-dcc1-4015-86b5-48a78b695204 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Unrescuing
Apr 12 04:04:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4f2f11fd-dcc1-4015-86b5-48a78b695204 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:04:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4f2f11fd-dcc1-4015-86b5-48a78b695204 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquired lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:04:41 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4f2f11fd-dcc1-4015-86b5-48a78b695204 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 04:04:41 user nova-compute[72081]: DEBUG nova.network.neutron [None req-4f2f11fd-dcc1-4015-86b5-48a78b695204 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Updating instance_info_cache with network_info: [{"id": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "address": "fa:16:3e:c4:e2:d7", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap16a3f234-3d", "ovs_interfaceid": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:04:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4f2f11fd-dcc1-4015-86b5-48a78b695204 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Releasing lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:04:41 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4f2f11fd-dcc1-4015-86b5-48a78b695204 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'flavor' on Instance uuid de40e137-84e3-4d85-895b-1f6da2673457 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:04:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:41 user nova-compute[72081]: DEBUG nova.compute.manager [req-d0d6a52a-7957-4af1-99aa-93208a13cf8a req-368437f8-a3a3-4eb8-a189-9edca2a831df service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received event network-vif-unplugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:04:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d0d6a52a-7957-4af1-99aa-93208a13cf8a req-368437f8-a3a3-4eb8-a189-9edca2a831df service nova] Acquiring lock "de40e137-84e3-4d85-895b-1f6da2673457-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:04:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d0d6a52a-7957-4af1-99aa-93208a13cf8a req-368437f8-a3a3-4eb8-a189-9edca2a831df service nova] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:04:41 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-d0d6a52a-7957-4af1-99aa-93208a13cf8a req-368437f8-a3a3-4eb8-a189-9edca2a831df service nova] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:04:41 user nova-compute[72081]: DEBUG nova.compute.manager [req-d0d6a52a-7957-4af1-99aa-93208a13cf8a req-368437f8-a3a3-4eb8-a189-9edca2a831df service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] No waiting events found dispatching network-vif-unplugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:04:41 user nova-compute[72081]: WARNING nova.compute.manager [req-d0d6a52a-7957-4af1-99aa-93208a13cf8a req-368437f8-a3a3-4eb8-a189-9edca2a831df service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received unexpected event network-vif-unplugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c for instance with vm_state rescued and task_state unrescuing.
Apr 12 04:04:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:41 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:42 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:42 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Instance destroyed successfully.
Apr 12 04:04:42 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:04:42 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 04:04:42 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Rebuilding the list of instances to heal {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 12 04:04:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:04:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:04:42 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 04:04:42 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lazy-loading 'info_cache' on Instance uuid de40e137-84e3-4d85-895b-1f6da2673457 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:04:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:43 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Updating instance_info_cache with network_info: [{"id": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "address": "fa:16:3e:c4:e2:d7", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap16a3f234-3d", "ovs_interfaceid": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:04:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:04:43 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 04:04:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:43 user nova-compute[72081]: DEBUG nova.compute.manager [req-03e5b478-a5fa-428f-ac77-da2550726ace req-45fa8556-3bcc-4147-9488-56fc15e2dcf8 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received event network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:04:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-03e5b478-a5fa-428f-ac77-da2550726ace req-45fa8556-3bcc-4147-9488-56fc15e2dcf8 service nova] Acquiring lock "de40e137-84e3-4d85-895b-1f6da2673457-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:04:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-03e5b478-a5fa-428f-ac77-da2550726ace req-45fa8556-3bcc-4147-9488-56fc15e2dcf8 service nova] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:04:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-03e5b478-a5fa-428f-ac77-da2550726ace req-45fa8556-3bcc-4147-9488-56fc15e2dcf8 service nova] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:04:43 user nova-compute[72081]: DEBUG nova.compute.manager [req-03e5b478-a5fa-428f-ac77-da2550726ace req-45fa8556-3bcc-4147-9488-56fc15e2dcf8 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] No waiting events found dispatching network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:04:43 user nova-compute[72081]: WARNING nova.compute.manager [req-03e5b478-a5fa-428f-ac77-da2550726ace req-45fa8556-3bcc-4147-9488-56fc15e2dcf8 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received unexpected event network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c for instance with vm_state rescued and task_state unrescuing.
Apr 12 04:04:43 user nova-compute[72081]: DEBUG nova.compute.manager [req-03e5b478-a5fa-428f-ac77-da2550726ace req-45fa8556-3bcc-4147-9488-56fc15e2dcf8 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received event network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:04:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-03e5b478-a5fa-428f-ac77-da2550726ace req-45fa8556-3bcc-4147-9488-56fc15e2dcf8 service nova] Acquiring lock "de40e137-84e3-4d85-895b-1f6da2673457-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:04:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-03e5b478-a5fa-428f-ac77-da2550726ace req-45fa8556-3bcc-4147-9488-56fc15e2dcf8 service nova] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:04:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-03e5b478-a5fa-428f-ac77-da2550726ace req-45fa8556-3bcc-4147-9488-56fc15e2dcf8 service nova] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:04:43 user nova-compute[72081]: DEBUG nova.compute.manager [req-03e5b478-a5fa-428f-ac77-da2550726ace req-45fa8556-3bcc-4147-9488-56fc15e2dcf8 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] No waiting events found dispatching network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:04:43 user nova-compute[72081]: WARNING nova.compute.manager [req-03e5b478-a5fa-428f-ac77-da2550726ace req-45fa8556-3bcc-4147-9488-56fc15e2dcf8 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received unexpected event network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c for instance with vm_state rescued and task_state unrescuing.
Apr 12 04:04:43 user nova-compute[72081]: DEBUG nova.compute.manager [req-03e5b478-a5fa-428f-ac77-da2550726ace req-45fa8556-3bcc-4147-9488-56fc15e2dcf8 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received event network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:04:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-03e5b478-a5fa-428f-ac77-da2550726ace req-45fa8556-3bcc-4147-9488-56fc15e2dcf8 service nova] Acquiring lock "de40e137-84e3-4d85-895b-1f6da2673457-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:04:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-03e5b478-a5fa-428f-ac77-da2550726ace req-45fa8556-3bcc-4147-9488-56fc15e2dcf8 service nova] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:04:43 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-03e5b478-a5fa-428f-ac77-da2550726ace req-45fa8556-3bcc-4147-9488-56fc15e2dcf8 service nova] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:04:43 user nova-compute[72081]: DEBUG nova.compute.manager [req-03e5b478-a5fa-428f-ac77-da2550726ace req-45fa8556-3bcc-4147-9488-56fc15e2dcf8 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] No waiting events found dispatching network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:04:43 user nova-compute[72081]: WARNING nova.compute.manager [req-03e5b478-a5fa-428f-ac77-da2550726ace req-45fa8556-3bcc-4147-9488-56fc15e2dcf8 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received unexpected event network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c for instance with vm_state rescued and task_state unrescuing.
Apr 12 04:04:45 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Removed pending event for de40e137-84e3-4d85-895b-1f6da2673457 due to event {{(pid=72081) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 12 04:04:45 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681272285.5173342, de40e137-84e3-4d85-895b-1f6da2673457 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 04:04:45 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] VM Resumed (Lifecycle Event)
Apr 12 04:04:45 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4f2f11fd-dcc1-4015-86b5-48a78b695204 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 04:04:45 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 04:04:45 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 04:04:45 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] During sync_power_state the instance has a pending task (unrescuing). Skip.
Apr 12 04:04:45 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681272285.5177193, de40e137-84e3-4d85-895b-1f6da2673457 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 04:04:45 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] VM Started (Lifecycle Event)
Apr 12 04:04:45 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 04:04:45 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Synchronizing instance power state after lifecycle event "Started"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 04:04:45 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:04:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:04:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:04:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:04:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG nova.objects.instance [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lazy-loading 'flavor' on Instance uuid 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.052s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:04:59 user nova-compute[72081]: INFO nova.compute.manager [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Attaching volume 463340ee-e367-446f-adf5-411c52acf103 to /dev/vdb
Apr 12 04:04:59 user nova-compute[72081]: DEBUG os_brick.utils [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d17db0a8-29bc-46eb-93e1-ccdb8d321b93]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[ec751754-3ae6-4893-a31a-d1c21de97e33]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[45f9a872-b9da-49c0-bec5-afa7a0466fb2]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f8e7cf51-a117-4c23-a959-9cb696173234]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 04:04:59 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 04:04:59 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1f732830-dc35-464e-81c5-12674750b05d]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG os_brick.utils [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] <== get_connector_properties: return (132ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 04:04:59 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Updating existing volume attachment record: 43c980d5-8c43-4a43-8db2-69840810b458 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 04:05:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'KpQmMUIY', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 04:05:03 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 04:05:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 04:05:03 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 04:05:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 04:05:03 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5b0e1e2a-53a3-4cec-827f-c9fd0ac4ea64]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 04:05:03 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f85f25a3-c1a4-44fe-b192-a003ffb1d9e6]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103] added
Apr 12 04:05:03 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[1de5db17-a663-47c8-be32-932793bb31b0]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7833d143-ef79-4176-b1a4-45add71a1e56]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v KpQmMUIY {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v KpQmMUIY" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[2db35281-c432-4e9d-ae92-797e71f0d44f]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'KpQmMUIY'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5b4a4b78-a089-4d19-a6a1-3da7a3f37bcf]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[353d56cc-2e6b-4ff8-bcef-55a4a13f76a3]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Apr 12 04:05:03 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Apr 12 04:05:03 user nova-compute[72081]:  {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 04:05:03 user nova-compute[72081]: WARNING os_brick.initiator.connectors.iscsi [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Apr 12 04:05:03 user nova-compute[72081]: 
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 -p 172.16.0.220:3260 --login" returned: 0 in 0.045s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5c1aa9f4-7415-473c-b805-9dd6537208a4]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103, portal: 172.16.0.220,3260]
Apr 12 04:05:03 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103, portal: 172.16.0.220,3260] successful.
Apr 12 04:05:03 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[afe510b2-499a-4f77-b5cd-22b630ae676b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[76f6003c-9658-4fbd-86e2-ab35b460d4e7]: (4, ('tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 (non-flash)
Apr 12 04:05:03 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsi session list stdout=tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 (non-flash)
Apr 12 04:05:03 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.258s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] HCTL ('33', '-', '-', 0) found on session 53 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c1d4f3fc-d1fd-4ad0-9d01-1031153cc7ca]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Searching for a device in session 53 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 04:05:03 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'KpQmMUIY', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 04:05:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.295s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 04:05:04 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] <== connect_volume: return (1297ms) {'type': 'block', 'scsi_wwn': '26363316539613462', 'path': '/dev/sdb'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 04:05:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26363316539613462', 'path': '/dev/sdb'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 04:05:04 user nova-compute[72081]: DEBUG nova.objects.instance [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lazy-loading 'flavor' on Instance uuid 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:05:04 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] attach device xml: <disk type="block" device="disk">
Apr 12 04:05:04 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 04:05:04 user nova-compute[72081]:   <source dev="/dev/sdb"/>
Apr 12 04:05:04 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 04:05:04 user nova-compute[72081]:   <serial>463340ee-e367-446f-adf5-411c52acf103</serial>
Apr 12 04:05:04 user nova-compute[72081]: </disk>
Apr 12 04:05:04 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 04:05:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 04:05:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 04:05:05 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] No VIF found with MAC fa:16:3e:b8:39:b0, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 04:05:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a4dc659b-78c7-4131-a91e-8e29895fe5fc tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.224s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:05:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:05:06 user nova-compute[72081]: INFO nova.compute.manager [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Detaching volume 463340ee-e367-446f-adf5-411c52acf103
Apr 12 04:05:06 user nova-compute[72081]: INFO nova.virt.block_device [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Attempting to driver detach volume 463340ee-e367-446f-adf5-411c52acf103 from mountpoint /dev/vdb
Apr 12 04:05:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Attempting to detach device vdb from instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 04:05:06 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 04:05:06 user nova-compute[72081]:   <source dev="/dev/sdb"/>
Apr 12 04:05:06 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 04:05:06 user nova-compute[72081]:   <serial>463340ee-e367-446f-adf5-411c52acf103</serial>
Apr 12 04:05:06 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 04:05:06 user nova-compute[72081]: </disk>
Apr 12 04:05:06 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 04:05:06 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Successfully detached device vdb from instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 from the persistent domain config.
Apr 12 04:05:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 from the live domain config. {{(pid=72081) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 04:05:06 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 04:05:06 user nova-compute[72081]:   <source dev="/dev/sdb"/>
Apr 12 04:05:06 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 04:05:06 user nova-compute[72081]:   <serial>463340ee-e367-446f-adf5-411c52acf103</serial>
Apr 12 04:05:06 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 04:05:06 user nova-compute[72081]: </disk>
Apr 12 04:05:06 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Received event <DeviceRemovedEvent: 1681272306.754304, 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72081) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 {{(pid=72081) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 12 04:05:06 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Successfully detached device vdb from instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 from the live domain config.
Apr 12 04:05:06 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'KpQmMUIY', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 04:05:06 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 04:05:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.029s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3c7ae940-7b4b-49c7-8cba-1e02f666f263]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 04:05:06 user nova-compute[72081]: No targets found.
Apr 12 04:05:06 user nova-compute[72081]: iSNS:
Apr 12 04:05:06 user nova-compute[72081]: No targets found.
Apr 12 04:05:06 user nova-compute[72081]: STATIC:
Apr 12 04:05:06 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103
Apr 12 04:05:06 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 04:05:06 user nova-compute[72081]:                 Iface Name: default
Apr 12 04:05:06 user nova-compute[72081]: FIRMWARE:
Apr 12 04:05:06 user nova-compute[72081]: No targets found.
Apr 12 04:05:06 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 04:05:06 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 04:05:06 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[3f51093c-038b-4299-bd95-cb7185f77a0b]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[4421adc5-a42a-4202-8f74-3489bc07b022]: (4, ('tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 (non-flash)
Apr 12 04:05:06 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsi session list stdout=tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 (non-flash)
Apr 12 04:05:06 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3f94119630>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103'): ({'sdb'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Removing single pathed devices sdb {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[82e371c7-0bb7-4a96-866a-3b4914f75d8d]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[fa4bf8f7-c71c-4318-894a-2d5c9b1da7c7]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Flushing IO for device /dev/sdb {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[0982c06d-3a67-42bd-8f22-ada98601ecb2]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 04:05:06 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.050s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[b50a9894-ebba-433d-a65d-9d804544abc4]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:05:07 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 04:05:07 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] SCSI volumes sdb have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 04:05:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 04:05:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[74e6f6f7-401c-4745-986e-52af7603ed33]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:05:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:05:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 -p 172.16.0.220:3260 --logout" returned: 0 in 0.070s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e405dc65-8f18-432f-b356-ef09aafd3762]: (4, ('Logging out of session [sid: 53, target: iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103, portal: 172.16.0.220,3260]\nLogout of [sid: 53, target: iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:05:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 53, target: iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103, portal: 172.16.0.220,3260]
Apr 12 04:05:07 user nova-compute[72081]: Logout of [sid: 53, target: iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103, portal: 172.16.0.220,3260] successful.
Apr 12 04:05:07 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:05:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:07 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:463340ee-e367-446f-adf5-411c52acf103 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:07 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[a8247390-5de7-4a11-b180-0798340c04e1]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:05:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:05:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.351s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 04:05:07 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] <== disconnect_volume: return (353ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 04:05:07 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 04:05:07 user nova-compute[72081]: DEBUG nova.objects.instance [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lazy-loading 'flavor' on Instance uuid 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:05:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-776a89e5-dd1c-48c3-8f98-3f668b1284ed tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.696s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:05:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:05:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:05:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:05:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:05:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:05:25 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:05:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:05:28 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:05:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:05:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:05:32 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:05:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:05:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:05:32 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:05:32 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 04:05:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:32 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.133s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:33 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:34 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:05:34 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:05:34 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8583MB free_disk=25.932422637939453GB free_vcpus=9 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 04:05:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:05:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:05:34 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance de40e137-84e3-4d85-895b-1f6da2673457 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:05:34 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 6cd92d70-a2af-4b2a-97c3-2981c157278d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:05:34 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:05:34 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 04:05:34 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=896MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 04:05:34 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 04:05:34 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 04:05:34 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 04:05:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.258s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:05:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:05:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:05:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:05:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:05:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:05:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:05:38 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:05:38 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:05:38 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:05:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 04:05:38 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:05:38 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:05:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:05:44 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:05:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 04:05:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-6cd92d70-a2af-4b2a-97c3-2981c157278d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:05:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-6cd92d70-a2af-4b2a-97c3-2981c157278d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:05:44 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 04:05:45 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Updating instance_info_cache with network_info: [{"id": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "address": "fa:16:3e:c0:fc:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapee1cfaf9-15", "ovs_interfaceid": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:05:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-6cd92d70-a2af-4b2a-97c3-2981c157278d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:05:45 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 04:05:45 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:05:45 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:05:45 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:05:45 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:05:45 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:05:45 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:05:50 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:05:52 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._run_image_cache_manager_pass {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:05:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "storage-registry-lock" by "nova.virt.storage_users.register_storage_use.<locals>.do_register_storage_use" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:05:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "storage-registry-lock" acquired by "nova.virt.storage_users.register_storage_use.<locals>.do_register_storage_use" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:05:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "storage-registry-lock" "released" by "nova.virt.storage_users.register_storage_use.<locals>.do_register_storage_use" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:05:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "storage-registry-lock" by "nova.virt.storage_users.get_storage_users.<locals>.do_get_storage_users" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:05:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "storage-registry-lock" acquired by "nova.virt.storage_users.get_storage_users.<locals>.do_get_storage_users" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:05:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "storage-registry-lock" "released" by "nova.virt.storage_users.get_storage_users.<locals>.do_get_storage_users" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:05:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Verify base images {{(pid=72081) _age_and_verify_cached_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:314}}
Apr 12 04:05:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Image id e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7 yields fingerprint 436a1a611c7731cbd94d56c4dfa86ba390d6d5df {{(pid=72081) _age_and_verify_cached_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:319}}
Apr 12 04:05:52 user nova-compute[72081]: INFO nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] image e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7 at (/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df): checking
Apr 12 04:05:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] image e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7 at (/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df): image is in use {{(pid=72081) _mark_in_use /opt/stack/nova/nova/virt/libvirt/imagecache.py:279}}
Apr 12 04:05:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Image id  yields fingerprint da39a3ee5e6b4b0d3255bfef95601890afd80709 {{(pid=72081) _age_and_verify_cached_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:319}}
Apr 12 04:05:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] de40e137-84e3-4d85-895b-1f6da2673457 is a valid instance name {{(pid=72081) _list_backing_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:126}}
Apr 12 04:05:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] de40e137-84e3-4d85-895b-1f6da2673457 has a disk file {{(pid=72081) _list_backing_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:129}}
Apr 12 04:05:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance de40e137-84e3-4d85-895b-1f6da2673457 is backed by 436a1a611c7731cbd94d56c4dfa86ba390d6d5df {{(pid=72081) _list_backing_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:141}}
Apr 12 04:05:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 is a valid instance name {{(pid=72081) _list_backing_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:126}}
Apr 12 04:05:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 has a disk file {{(pid=72081) _list_backing_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:129}}
Apr 12 04:05:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 is backed by 436a1a611c7731cbd94d56c4dfa86ba390d6d5df {{(pid=72081) _list_backing_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:141}}
Apr 12 04:05:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] 6cd92d70-a2af-4b2a-97c3-2981c157278d is a valid instance name {{(pid=72081) _list_backing_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:126}}
Apr 12 04:05:52 user nova-compute[72081]: DEBUG nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] 6cd92d70-a2af-4b2a-97c3-2981c157278d has a disk file {{(pid=72081) _list_backing_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:129}}
Apr 12 04:05:52 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:05:53 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:05:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 6cd92d70-a2af-4b2a-97c3-2981c157278d is backed by 436a1a611c7731cbd94d56c4dfa86ba390d6d5df {{(pid=72081) _list_backing_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:141}}
Apr 12 04:05:53 user nova-compute[72081]: WARNING nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Unknown base file: /opt/stack/data/nova/instances/_base/1b8169da5e88f5971780e0a8c6222f174cec0901
Apr 12 04:05:53 user nova-compute[72081]: WARNING nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Unknown base file: /opt/stack/data/nova/instances/_base/f8989bf22d0757204243efbf1bf4173532921efe
Apr 12 04:05:53 user nova-compute[72081]: WARNING nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Unknown base file: /opt/stack/data/nova/instances/_base/20a9ef7c61ef4dc87afe749fc5ad5499efda7617
Apr 12 04:05:53 user nova-compute[72081]: WARNING nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Unknown base file: /opt/stack/data/nova/instances/_base/df2c896d566356f451d4aa39269f45190fe99704
Apr 12 04:05:53 user nova-compute[72081]: WARNING nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Unknown base file: /opt/stack/data/nova/instances/_base/97536260b2fcf56105eaaf8a9865fca3acbfc0a0
Apr 12 04:05:53 user nova-compute[72081]: WARNING nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Unknown base file: /opt/stack/data/nova/instances/_base/52bac5158985566ef9844ce516d5fe47fe83736c
Apr 12 04:05:53 user nova-compute[72081]: WARNING nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Unknown base file: /opt/stack/data/nova/instances/_base/c24b9043f1dedaa2da7a42314522ae21f8755c66
Apr 12 04:05:53 user nova-compute[72081]: WARNING nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Unknown base file: /opt/stack/data/nova/instances/_base/ceec718a3559ac0750129010fcadb4c577075d0b
Apr 12 04:05:53 user nova-compute[72081]: WARNING nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Unknown base file: /opt/stack/data/nova/instances/_base/3eb9c7fe4cd49992d2a057f19ea3825baa4d1d9e
Apr 12 04:05:53 user nova-compute[72081]: INFO nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Active base files: /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df
Apr 12 04:05:53 user nova-compute[72081]: INFO nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Removable base files: /opt/stack/data/nova/instances/_base/1b8169da5e88f5971780e0a8c6222f174cec0901 /opt/stack/data/nova/instances/_base/f8989bf22d0757204243efbf1bf4173532921efe /opt/stack/data/nova/instances/_base/20a9ef7c61ef4dc87afe749fc5ad5499efda7617 /opt/stack/data/nova/instances/_base/df2c896d566356f451d4aa39269f45190fe99704 /opt/stack/data/nova/instances/_base/97536260b2fcf56105eaaf8a9865fca3acbfc0a0 /opt/stack/data/nova/instances/_base/52bac5158985566ef9844ce516d5fe47fe83736c /opt/stack/data/nova/instances/_base/c24b9043f1dedaa2da7a42314522ae21f8755c66 /opt/stack/data/nova/instances/_base/ceec718a3559ac0750129010fcadb4c577075d0b /opt/stack/data/nova/instances/_base/3eb9c7fe4cd49992d2a057f19ea3825baa4d1d9e
Apr 12 04:05:53 user nova-compute[72081]: INFO nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/1b8169da5e88f5971780e0a8c6222f174cec0901
Apr 12 04:05:53 user nova-compute[72081]: INFO nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/f8989bf22d0757204243efbf1bf4173532921efe
Apr 12 04:05:53 user nova-compute[72081]: INFO nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/20a9ef7c61ef4dc87afe749fc5ad5499efda7617
Apr 12 04:05:53 user nova-compute[72081]: INFO nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/df2c896d566356f451d4aa39269f45190fe99704
Apr 12 04:05:53 user nova-compute[72081]: INFO nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/97536260b2fcf56105eaaf8a9865fca3acbfc0a0
Apr 12 04:05:53 user nova-compute[72081]: INFO nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/52bac5158985566ef9844ce516d5fe47fe83736c
Apr 12 04:05:53 user nova-compute[72081]: INFO nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/c24b9043f1dedaa2da7a42314522ae21f8755c66
Apr 12 04:05:53 user nova-compute[72081]: INFO nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/ceec718a3559ac0750129010fcadb4c577075d0b
Apr 12 04:05:53 user nova-compute[72081]: INFO nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/3eb9c7fe4cd49992d2a057f19ea3825baa4d1d9e
Apr 12 04:05:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Verification complete {{(pid=72081) _age_and_verify_cached_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:350}}
Apr 12 04:05:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Verify swap images {{(pid=72081) _age_and_verify_swap_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:299}}
Apr 12 04:05:53 user nova-compute[72081]: DEBUG nova.virt.libvirt.imagecache [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Verify ephemeral images {{(pid=72081) _age_and_verify_ephemeral_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:284}}
Apr 12 04:05:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:05:55 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:06:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:06:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:06:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:06:00 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:06:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:06:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:06:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:06:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-dd8bea29-07b4-40b7-bba9-5810e8efb5f4 req-e42d025d-07a3-429a-91d7-f8b8aef43d3e service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Received event network-changed-c83728df-a614-42d7-a164-d5d5e0373838 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:06:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-dd8bea29-07b4-40b7-bba9-5810e8efb5f4 req-e42d025d-07a3-429a-91d7-f8b8aef43d3e service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Refreshing instance network info cache due to event network-changed-c83728df-a614-42d7-a164-d5d5e0373838. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 04:06:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-dd8bea29-07b4-40b7-bba9-5810e8efb5f4 req-e42d025d-07a3-429a-91d7-f8b8aef43d3e service nova] Acquiring lock "refresh_cache-7a6fb26c-9c8f-4be7-b16f-8662eed2e489" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:06:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-dd8bea29-07b4-40b7-bba9-5810e8efb5f4 req-e42d025d-07a3-429a-91d7-f8b8aef43d3e service nova] Acquired lock "refresh_cache-7a6fb26c-9c8f-4be7-b16f-8662eed2e489" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:06:07 user nova-compute[72081]: DEBUG nova.network.neutron [req-dd8bea29-07b4-40b7-bba9-5810e8efb5f4 req-e42d025d-07a3-429a-91d7-f8b8aef43d3e service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Refreshing network info cache for port c83728df-a614-42d7-a164-d5d5e0373838 {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 04:06:07 user nova-compute[72081]: DEBUG nova.network.neutron [req-dd8bea29-07b4-40b7-bba9-5810e8efb5f4 req-e42d025d-07a3-429a-91d7-f8b8aef43d3e service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Updated VIF entry in instance network info cache for port c83728df-a614-42d7-a164-d5d5e0373838. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 04:06:07 user nova-compute[72081]: DEBUG nova.network.neutron [req-dd8bea29-07b4-40b7-bba9-5810e8efb5f4 req-e42d025d-07a3-429a-91d7-f8b8aef43d3e service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Updating instance_info_cache with network_info: [{"id": "c83728df-a614-42d7-a164-d5d5e0373838", "address": "fa:16:3e:b8:39:b0", "network": {"id": "1298d27c-9dfa-46e2-b941-024dd99ead79", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-1228939766-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01223543586e474b875e7dbf2c78ef87", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc83728df-a6", "ovs_interfaceid": "c83728df-a614-42d7-a164-d5d5e0373838", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:06:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-dd8bea29-07b4-40b7-bba9-5810e8efb5f4 req-e42d025d-07a3-429a-91d7-f8b8aef43d3e service nova] Releasing lock "refresh_cache-7a6fb26c-9c8f-4be7-b16f-8662eed2e489" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:06:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4a3a1277-86e9-478f-a012-acd090ae130c tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:06:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4a3a1277-86e9-478f-a012-acd090ae130c tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:06:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4a3a1277-86e9-478f-a012-acd090ae130c tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:06:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4a3a1277-86e9-478f-a012-acd090ae130c tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:06:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4a3a1277-86e9-478f-a012-acd090ae130c tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:06:09 user nova-compute[72081]: INFO nova.compute.manager [None req-4a3a1277-86e9-478f-a012-acd090ae130c tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Terminating instance
Apr 12 04:06:09 user nova-compute[72081]: DEBUG nova.compute.manager [None req-4a3a1277-86e9-478f-a012-acd090ae130c tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 04:06:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:10 user nova-compute[72081]: DEBUG nova.compute.manager [req-7af60b53-8fdc-48a4-91fa-d54b256c4890 req-9d8c1c6f-ba00-4243-8ce6-8bfd8f04a13c service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Received event network-vif-unplugged-c83728df-a614-42d7-a164-d5d5e0373838 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:06:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7af60b53-8fdc-48a4-91fa-d54b256c4890 req-9d8c1c6f-ba00-4243-8ce6-8bfd8f04a13c service nova] Acquiring lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:06:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7af60b53-8fdc-48a4-91fa-d54b256c4890 req-9d8c1c6f-ba00-4243-8ce6-8bfd8f04a13c service nova] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:06:10 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-7af60b53-8fdc-48a4-91fa-d54b256c4890 req-9d8c1c6f-ba00-4243-8ce6-8bfd8f04a13c service nova] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:06:10 user nova-compute[72081]: DEBUG nova.compute.manager [req-7af60b53-8fdc-48a4-91fa-d54b256c4890 req-9d8c1c6f-ba00-4243-8ce6-8bfd8f04a13c service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] No waiting events found dispatching network-vif-unplugged-c83728df-a614-42d7-a164-d5d5e0373838 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:06:10 user nova-compute[72081]: DEBUG nova.compute.manager [req-7af60b53-8fdc-48a4-91fa-d54b256c4890 req-9d8c1c6f-ba00-4243-8ce6-8bfd8f04a13c service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Received event network-vif-unplugged-c83728df-a614-42d7-a164-d5d5e0373838 for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 04:06:10 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Instance destroyed successfully.
Apr 12 04:06:10 user nova-compute[72081]: DEBUG nova.objects.instance [None req-4a3a1277-86e9-478f-a012-acd090ae130c tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lazy-loading 'resources' on Instance uuid 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:06:10 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-4a3a1277-86e9-478f-a012-acd090ae130c tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:59:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-SnapshotDataIntegrityTests-server-1263744278',display_name='tempest-SnapshotDataIntegrityTests-server-1263744278',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-snapshotdataintegritytests-server-1263744278',id=46,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNS5zNm2ebJK6LG9OrL+ojzX+EwxzS/Yi0eAvnaL94lZQddER4Le1QtAFpoIWWwQszoaOHdLJaBYOeLou7nEAtCK/7ohtIg1OU5j5fJMB0C0xARTUV1WXA8kI8LAoQVCtA==',key_name='tempest-SnapshotDataIntegrityTests-272659456',keypairs=<?>,launch_index=0,launched_at=2023-04-12T04:00:05Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='01223543586e474b875e7dbf2c78ef87',ramdisk_id='',reservation_id='r-6z106nur',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-SnapshotDataIntegrityTests-956619934',owner_user_name='tempest-SnapshotDataIntegrityTests-956619934-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T04:00:06Z,user_data=None,user_id='dc05e2e474814697920e72d50883e263',uuid=7a6fb26c-9c8f-4be7-b16f-8662eed2e489,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "c83728df-a614-42d7-a164-d5d5e0373838", "address": "fa:16:3e:b8:39:b0", "network": {"id": "1298d27c-9dfa-46e2-b941-024dd99ead79", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-1228939766-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01223543586e474b875e7dbf2c78ef87", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc83728df-a6", "ovs_interfaceid": "c83728df-a614-42d7-a164-d5d5e0373838", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 04:06:10 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4a3a1277-86e9-478f-a012-acd090ae130c tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Converting VIF {"id": "c83728df-a614-42d7-a164-d5d5e0373838", "address": "fa:16:3e:b8:39:b0", "network": {"id": "1298d27c-9dfa-46e2-b941-024dd99ead79", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-1228939766-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "01223543586e474b875e7dbf2c78ef87", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc83728df-a6", "ovs_interfaceid": "c83728df-a614-42d7-a164-d5d5e0373838", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 04:06:10 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-4a3a1277-86e9-478f-a012-acd090ae130c tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:b8:39:b0,bridge_name='br-int',has_traffic_filtering=True,id=c83728df-a614-42d7-a164-d5d5e0373838,network=Network(1298d27c-9dfa-46e2-b941-024dd99ead79),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc83728df-a6') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 04:06:10 user nova-compute[72081]: DEBUG os_vif [None req-4a3a1277-86e9-478f-a012-acd090ae130c tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:b8:39:b0,bridge_name='br-int',has_traffic_filtering=True,id=c83728df-a614-42d7-a164-d5d5e0373838,network=Network(1298d27c-9dfa-46e2-b941-024dd99ead79),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc83728df-a6') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 04:06:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapc83728df-a6, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 04:06:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:10 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:06:10 user nova-compute[72081]: INFO os_vif [None req-4a3a1277-86e9-478f-a012-acd090ae130c tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:b8:39:b0,bridge_name='br-int',has_traffic_filtering=True,id=c83728df-a614-42d7-a164-d5d5e0373838,network=Network(1298d27c-9dfa-46e2-b941-024dd99ead79),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc83728df-a6')
Apr 12 04:06:10 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-4a3a1277-86e9-478f-a012-acd090ae130c tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Deleting instance files /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489_del
Apr 12 04:06:10 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-4a3a1277-86e9-478f-a012-acd090ae130c tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Deletion of /opt/stack/data/nova/instances/7a6fb26c-9c8f-4be7-b16f-8662eed2e489_del complete
Apr 12 04:06:10 user nova-compute[72081]: INFO nova.compute.manager [None req-4a3a1277-86e9-478f-a012-acd090ae130c tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Took 0.86 seconds to destroy the instance on the hypervisor.
Apr 12 04:06:10 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-4a3a1277-86e9-478f-a012-acd090ae130c tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 04:06:10 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 04:06:10 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 04:06:11 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:06:11 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Took 0.59 seconds to deallocate network for instance.
Apr 12 04:06:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4a3a1277-86e9-478f-a012-acd090ae130c tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:06:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4a3a1277-86e9-478f-a012-acd090ae130c tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:06:11 user nova-compute[72081]: DEBUG nova.compute.manager [req-63dd59f9-29cb-4f55-bd52-0a83e8f84f6e req-d2ab4da9-1572-486f-8abd-992af1507d09 service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Received event network-vif-deleted-c83728df-a614-42d7-a164-d5d5e0373838 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:06:11 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-4a3a1277-86e9-478f-a012-acd090ae130c tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 04:06:11 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-4a3a1277-86e9-478f-a012-acd090ae130c tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 04:06:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4a3a1277-86e9-478f-a012-acd090ae130c tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.166s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:06:11 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-4a3a1277-86e9-478f-a012-acd090ae130c tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Deleted allocations for instance 7a6fb26c-9c8f-4be7-b16f-8662eed2e489
Apr 12 04:06:11 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-4a3a1277-86e9-478f-a012-acd090ae130c tempest-SnapshotDataIntegrityTests-956619934 tempest-SnapshotDataIntegrityTests-956619934-project-member] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.795s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:06:12 user nova-compute[72081]: DEBUG nova.compute.manager [req-a041c1be-b685-445d-a351-cf232ac3679f req-8d5b4791-891f-44bd-8ced-2dfb5aaa66f9 service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Received event network-vif-plugged-c83728df-a614-42d7-a164-d5d5e0373838 {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:06:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a041c1be-b685-445d-a351-cf232ac3679f req-8d5b4791-891f-44bd-8ced-2dfb5aaa66f9 service nova] Acquiring lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:06:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a041c1be-b685-445d-a351-cf232ac3679f req-8d5b4791-891f-44bd-8ced-2dfb5aaa66f9 service nova] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:06:12 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-a041c1be-b685-445d-a351-cf232ac3679f req-8d5b4791-891f-44bd-8ced-2dfb5aaa66f9 service nova] Lock "7a6fb26c-9c8f-4be7-b16f-8662eed2e489-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:06:12 user nova-compute[72081]: DEBUG nova.compute.manager [req-a041c1be-b685-445d-a351-cf232ac3679f req-8d5b4791-891f-44bd-8ced-2dfb5aaa66f9 service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] No waiting events found dispatching network-vif-plugged-c83728df-a614-42d7-a164-d5d5e0373838 {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:06:12 user nova-compute[72081]: WARNING nova.compute.manager [req-a041c1be-b685-445d-a351-cf232ac3679f req-8d5b4791-891f-44bd-8ced-2dfb5aaa66f9 service nova] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Received unexpected event network-vif-plugged-c83728df-a614-42d7-a164-d5d5e0373838 for instance with vm_state deleted and task_state None.
Apr 12 04:06:15 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:20 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:25 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681272370.4540296, 7a6fb26c-9c8f-4be7-b16f-8662eed2e489 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 04:06:25 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] VM Stopped (Lifecycle Event)
Apr 12 04:06:25 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9b7b3036-defa-4ebb-947d-a1795b936d2c None None] [instance: 7a6fb26c-9c8f-4be7-b16f-8662eed2e489] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 04:06:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:06:28 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:06:29 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:06:30 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:34 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:06:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:06:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:06:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:06:34 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 04:06:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:06:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:06:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:06:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:06:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:06:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:06:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:06:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:06:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:06:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:06:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:06:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:06:36 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:06:36 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:06:36 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8693MB free_disk=25.950836181640625GB free_vcpus=10 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG nova.compute.manager [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Starting instance... {{(pid=72081) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance de40e137-84e3-4d85-895b-1f6da2673457 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 6cd92d70-a2af-4b2a-97c3-2981c157278d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 has been scheduled to this compute host, the scheduler has made an allocation against this compute node but the instance has yet to start. Skipping heal of allocation: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1689}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 2 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=768MB phys_disk=40GB used_disk=2GB total_vcpus=12 used_vcpus=2 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Refreshing inventories for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Updating ProviderTree inventory for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Updating inventory in ProviderTree for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Refreshing aggregate associations for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6, aggregates: None {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Refreshing trait associations for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6, traits: COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_USB,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_RESCUE_BFV,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE41,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_STORAGE_BUS_FDC,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_STORAGE_BUS_IDE,COMPUTE_TRUSTED_CERTS,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_GRAPHICS_MODEL_CIRRUS,HW_CPU_X86_SSSE3,HW_CPU_X86_SSE,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_EXTEND,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_STORAGE_BUS_SATA,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_GRAPHICS_MODEL_QXL,HW_CPU_X86_SSE42,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_ACCELERATORS,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,HW_CPU_X86_MMX,COMPUTE_NODE,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_DEVICE_TAGGING,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_VIOMMU_MODEL_AUTO {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.427s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.278s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72081) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 12 04:06:36 user nova-compute[72081]: INFO nova.compute.claims [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Claim successful on node user
Apr 12 04:06:36 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.261s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG nova.compute.manager [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Start building networks asynchronously for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG nova.compute.manager [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Allocating IP information in the background. {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG nova.network.neutron [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] allocate_for_instance() {{(pid=72081) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 12 04:06:36 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 12 04:06:36 user nova-compute[72081]: DEBUG nova.compute.manager [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Start building block device mappings for instance. {{(pid=72081) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG nova.policy [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '44a0206f1323440db4f8411a3c3c088d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a6a8a4f5d59e479f9ca7b73a9e270c4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72081) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG nova.compute.manager [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Start spawning the instance on the hypervisor. {{(pid=72081) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Creating instance directory {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 12 04:06:36 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Creating image(s)
Apr 12 04:06:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "/opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "/opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "/opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:06:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:06:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:06:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:06:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:06:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:06:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:06:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk 1073741824 {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:06:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk 1073741824" returned: 0 in 0.052s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:06:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.199s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:06:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:06:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:06:37 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Checking if we can resize image /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk. size=1073741824 {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 12 04:06:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:06:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:06:37 user nova-compute[72081]: DEBUG nova.virt.disk.api [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Cannot resize image /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk to a smaller size. {{(pid=72081) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 12 04:06:37 user nova-compute[72081]: DEBUG nova.objects.instance [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'migration_context' on Instance uuid 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:06:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 04:06:37 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Ensure instance console log exists: /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/console.log {{(pid=72081) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 12 04:06:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:06:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:06:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:06:37 user nova-compute[72081]: DEBUG nova.network.neutron [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Successfully created port: b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.network.neutron [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Successfully updated port: b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquired lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.network.neutron [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-dd381844-efcb-47bf-9edb-f31e1964ec65 req-bf65daae-452d-4c75-adc4-29ed23820dc6 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-changed-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.compute.manager [req-dd381844-efcb-47bf-9edb-f31e1964ec65 req-bf65daae-452d-4c75-adc4-29ed23820dc6 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Refreshing instance network info cache due to event network-changed-b2759903-00b2-4ce7-8693-e93f30bb850c. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-dd381844-efcb-47bf-9edb-f31e1964ec65 req-bf65daae-452d-4c75-adc4-29ed23820dc6 service nova] Acquiring lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.network.neutron [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Instance cache missing network info. {{(pid=72081) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.network.neutron [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Updating instance_info_cache with network_info: [{"id": "b2759903-00b2-4ce7-8693-e93f30bb850c", "address": "fa:16:3e:b6:f9:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb2759903-00", "ovs_interfaceid": "b2759903-00b2-4ce7-8693-e93f30bb850c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Releasing lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.compute.manager [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Instance network_info: |[{"id": "b2759903-00b2-4ce7-8693-e93f30bb850c", "address": "fa:16:3e:b6:f9:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb2759903-00", "ovs_interfaceid": "b2759903-00b2-4ce7-8693-e93f30bb850c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72081) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-dd381844-efcb-47bf-9edb-f31e1964ec65 req-bf65daae-452d-4c75-adc4-29ed23820dc6 service nova] Acquired lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.network.neutron [req-dd381844-efcb-47bf-9edb-f31e1964ec65 req-bf65daae-452d-4c75-adc4-29ed23820dc6 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Refreshing network info cache for port b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Start _get_guest_xml network_info=[{"id": "b2759903-00b2-4ce7-8693-e93f30bb850c", "address": "fa:16:3e:b6:f9:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb2759903-00", "ovs_interfaceid": "b2759903-00b2-4ce7-8693-e93f30bb850c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'disk_bus': 'virtio', 'encryption_secret_uuid': None, 'size': 0, 'encrypted': False, 'boot_index': 0, 'encryption_options': None, 'device_name': '/dev/vda', 'guest_format': None, 'encryption_format': None, 'device_type': 'disk', 'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 04:06:38 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:06:38 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T04:06:35Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-1315935319',display_name='tempest-ServerRescueNegativeTestJSON-server-1315935319',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-1315935319',id=47,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLXMs69H3aVwHvUXehqGeCn+vQJTwljOWf0Mp9khd00+UAOaU6NKxayjTMFZ3zU9NHkMqh5MJ76PIyP2BqxywFz+m9V79UNQ6CkqenIlufndQEJ4s9KnY6yNND9omt/Bww==',key_name='tempest-keypair-469921698',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a6a8a4f5d59e479f9ca7b73a9e270c4b',ramdisk_id='',reservation_id='r-dat2ec2g',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-529226261',owner_user_name='tempest-ServerRescueNegativeTestJSON-529226261-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T04:06:37Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='44a0206f1323440db4f8411a3c3c088d',uuid=24cfaf66-a8ca-4dc3-8b83-1fe7de583f43,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "b2759903-00b2-4ce7-8693-e93f30bb850c", "address": "fa:16:3e:b6:f9:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb2759903-00", "ovs_interfaceid": "b2759903-00b2-4ce7-8693-e93f30bb850c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Converting VIF {"id": "b2759903-00b2-4ce7-8693-e93f30bb850c", "address": "fa:16:3e:b6:f9:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb2759903-00", "ovs_interfaceid": "b2759903-00b2-4ce7-8693-e93f30bb850c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b6:f9:3b,bridge_name='br-int',has_traffic_filtering=True,id=b2759903-00b2-4ce7-8693-e93f30bb850c,network=Network(311bda05-075c-4bb3-9469-b5117dd33353),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb2759903-00') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.objects.instance [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'pci_devices' on Instance uuid 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] End _get_guest_xml xml=<domain type="kvm">
Apr 12 04:06:38 user nova-compute[72081]:   <uuid>24cfaf66-a8ca-4dc3-8b83-1fe7de583f43</uuid>
Apr 12 04:06:38 user nova-compute[72081]:   <name>instance-0000002f</name>
Apr 12 04:06:38 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 04:06:38 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 04:06:38 user nova-compute[72081]:   <metadata>
Apr 12 04:06:38 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 04:06:38 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 04:06:38 user nova-compute[72081]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-1315935319</nova:name>
Apr 12 04:06:38 user nova-compute[72081]:       <nova:creationTime>2023-04-12 04:06:38</nova:creationTime>
Apr 12 04:06:38 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 04:06:38 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 04:06:38 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 04:06:38 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 04:06:38 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 04:06:38 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 04:06:38 user nova-compute[72081]:       </nova:flavor>
Apr 12 04:06:38 user nova-compute[72081]:       <nova:owner>
Apr 12 04:06:38 user nova-compute[72081]:         <nova:user uuid="44a0206f1323440db4f8411a3c3c088d">tempest-ServerRescueNegativeTestJSON-529226261-project-member</nova:user>
Apr 12 04:06:38 user nova-compute[72081]:         <nova:project uuid="a6a8a4f5d59e479f9ca7b73a9e270c4b">tempest-ServerRescueNegativeTestJSON-529226261</nova:project>
Apr 12 04:06:38 user nova-compute[72081]:       </nova:owner>
Apr 12 04:06:38 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 04:06:38 user nova-compute[72081]:       <nova:ports>
Apr 12 04:06:38 user nova-compute[72081]:         <nova:port uuid="b2759903-00b2-4ce7-8693-e93f30bb850c">
Apr 12 04:06:38 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.11" ipVersion="4"/>
Apr 12 04:06:38 user nova-compute[72081]:         </nova:port>
Apr 12 04:06:38 user nova-compute[72081]:       </nova:ports>
Apr 12 04:06:38 user nova-compute[72081]:     </nova:instance>
Apr 12 04:06:38 user nova-compute[72081]:   </metadata>
Apr 12 04:06:38 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 04:06:38 user nova-compute[72081]:     <system>
Apr 12 04:06:38 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 04:06:38 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 04:06:38 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 04:06:38 user nova-compute[72081]:       <entry name="serial">24cfaf66-a8ca-4dc3-8b83-1fe7de583f43</entry>
Apr 12 04:06:38 user nova-compute[72081]:       <entry name="uuid">24cfaf66-a8ca-4dc3-8b83-1fe7de583f43</entry>
Apr 12 04:06:38 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 04:06:38 user nova-compute[72081]:     </system>
Apr 12 04:06:38 user nova-compute[72081]:   </sysinfo>
Apr 12 04:06:38 user nova-compute[72081]:   <os>
Apr 12 04:06:38 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 04:06:38 user nova-compute[72081]:     <boot dev="hd"/>
Apr 12 04:06:38 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 04:06:38 user nova-compute[72081]:   </os>
Apr 12 04:06:38 user nova-compute[72081]:   <features>
Apr 12 04:06:38 user nova-compute[72081]:     <acpi/>
Apr 12 04:06:38 user nova-compute[72081]:     <apic/>
Apr 12 04:06:38 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 04:06:38 user nova-compute[72081]:   </features>
Apr 12 04:06:38 user nova-compute[72081]:   <clock offset="utc">
Apr 12 04:06:38 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 04:06:38 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 04:06:38 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 04:06:38 user nova-compute[72081]:   </clock>
Apr 12 04:06:38 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 04:06:38 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 04:06:38 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 04:06:38 user nova-compute[72081]:   </cpu>
Apr 12 04:06:38 user nova-compute[72081]:   <devices>
Apr 12 04:06:38 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 04:06:38 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 04:06:38 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk"/>
Apr 12 04:06:38 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 04:06:38 user nova-compute[72081]:     </disk>
Apr 12 04:06:38 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 04:06:38 user nova-compute[72081]:       <mac address="fa:16:3e:b6:f9:3b"/>
Apr 12 04:06:38 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 04:06:38 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 04:06:38 user nova-compute[72081]:       <target dev="tapb2759903-00"/>
Apr 12 04:06:38 user nova-compute[72081]:     </interface>
Apr 12 04:06:38 user nova-compute[72081]:     <serial type="pty">
Apr 12 04:06:38 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/console.log" append="off"/>
Apr 12 04:06:38 user nova-compute[72081]:     </serial>
Apr 12 04:06:38 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 04:06:38 user nova-compute[72081]:     <video>
Apr 12 04:06:38 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 04:06:38 user nova-compute[72081]:     </video>
Apr 12 04:06:38 user nova-compute[72081]:     <rng model="virtio">
Apr 12 04:06:38 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 04:06:38 user nova-compute[72081]:     </rng>
Apr 12 04:06:38 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 04:06:38 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 04:06:38 user nova-compute[72081]:       <stats period="10"/>
Apr 12 04:06:38 user nova-compute[72081]:     </memballoon>
Apr 12 04:06:38 user nova-compute[72081]:   </devices>
Apr 12 04:06:38 user nova-compute[72081]: </domain>
Apr 12 04:06:38 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T04:06:35Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-1315935319',display_name='tempest-ServerRescueNegativeTestJSON-server-1315935319',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-1315935319',id=47,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLXMs69H3aVwHvUXehqGeCn+vQJTwljOWf0Mp9khd00+UAOaU6NKxayjTMFZ3zU9NHkMqh5MJ76PIyP2BqxywFz+m9V79UNQ6CkqenIlufndQEJ4s9KnY6yNND9omt/Bww==',key_name='tempest-keypair-469921698',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a6a8a4f5d59e479f9ca7b73a9e270c4b',ramdisk_id='',reservation_id='r-dat2ec2g',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-529226261',owner_user_name='tempest-ServerRescueNegativeTestJSON-529226261-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T04:06:37Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='44a0206f1323440db4f8411a3c3c088d',uuid=24cfaf66-a8ca-4dc3-8b83-1fe7de583f43,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "b2759903-00b2-4ce7-8693-e93f30bb850c", "address": "fa:16:3e:b6:f9:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb2759903-00", "ovs_interfaceid": "b2759903-00b2-4ce7-8693-e93f30bb850c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Converting VIF {"id": "b2759903-00b2-4ce7-8693-e93f30bb850c", "address": "fa:16:3e:b6:f9:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb2759903-00", "ovs_interfaceid": "b2759903-00b2-4ce7-8693-e93f30bb850c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b6:f9:3b,bridge_name='br-int',has_traffic_filtering=True,id=b2759903-00b2-4ce7-8693-e93f30bb850c,network=Network(311bda05-075c-4bb3-9469-b5117dd33353),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb2759903-00') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG os_vif [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:b6:f9:3b,bridge_name='br-int',has_traffic_filtering=True,id=b2759903-00b2-4ce7-8693-e93f30bb850c,network=Network(311bda05-075c-4bb3-9469-b5117dd33353),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb2759903-00') {{(pid=72081) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapb2759903-00, may_exist=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapb2759903-00, col_values=(('external_ids', {'iface-id': 'b2759903-00b2-4ce7-8693-e93f30bb850c', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:b6:f9:3b', 'vm-uuid': '24cfaf66-a8ca-4dc3-8b83-1fe7de583f43'}),)) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:38 user nova-compute[72081]: INFO os_vif [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:b6:f9:3b,bridge_name='br-int',has_traffic_filtering=True,id=b2759903-00b2-4ce7-8693-e93f30bb850c,network=Network(311bda05-075c-4bb3-9469-b5117dd33353),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb2759903-00')
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 04:06:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] No VIF found with MAC fa:16:3e:b6:f9:3b, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 04:06:39 user nova-compute[72081]: DEBUG nova.network.neutron [req-dd381844-efcb-47bf-9edb-f31e1964ec65 req-bf65daae-452d-4c75-adc4-29ed23820dc6 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Updated VIF entry in instance network info cache for port b2759903-00b2-4ce7-8693-e93f30bb850c. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 04:06:39 user nova-compute[72081]: DEBUG nova.network.neutron [req-dd381844-efcb-47bf-9edb-f31e1964ec65 req-bf65daae-452d-4c75-adc4-29ed23820dc6 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Updating instance_info_cache with network_info: [{"id": "b2759903-00b2-4ce7-8693-e93f30bb850c", "address": "fa:16:3e:b6:f9:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb2759903-00", "ovs_interfaceid": "b2759903-00b2-4ce7-8693-e93f30bb850c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:06:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-dd381844-efcb-47bf-9edb-f31e1964ec65 req-bf65daae-452d-4c75-adc4-29ed23820dc6 service nova] Releasing lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:06:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:40 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:40 user nova-compute[72081]: DEBUG nova.compute.manager [req-9949e9e1-ef25-40e9-b37e-30ae113de1f6 req-9ec7300a-e219-4482-a6bf-52bd1f29cb56 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:06:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9949e9e1-ef25-40e9-b37e-30ae113de1f6 req-9ec7300a-e219-4482-a6bf-52bd1f29cb56 service nova] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:06:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9949e9e1-ef25-40e9-b37e-30ae113de1f6 req-9ec7300a-e219-4482-a6bf-52bd1f29cb56 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:06:40 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-9949e9e1-ef25-40e9-b37e-30ae113de1f6 req-9ec7300a-e219-4482-a6bf-52bd1f29cb56 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:06:40 user nova-compute[72081]: DEBUG nova.compute.manager [req-9949e9e1-ef25-40e9-b37e-30ae113de1f6 req-9ec7300a-e219-4482-a6bf-52bd1f29cb56 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] No waiting events found dispatching network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:06:40 user nova-compute[72081]: WARNING nova.compute.manager [req-9949e9e1-ef25-40e9-b37e-30ae113de1f6 req-9ec7300a-e219-4482-a6bf-52bd1f29cb56 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received unexpected event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c for instance with vm_state building and task_state spawning.
Apr 12 04:06:40 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:06:42 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681272402.3378756, 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 04:06:42 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] VM Resumed (Lifecycle Event)
Apr 12 04:06:42 user nova-compute[72081]: DEBUG nova.compute.manager [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Instance event wait completed in 0 seconds for  {{(pid=72081) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 12 04:06:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Guest created on hypervisor {{(pid=72081) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 12 04:06:42 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Instance spawned successfully.
Apr 12 04:06:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 12 04:06:42 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 04:06:42 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 04:06:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Found default for hw_cdrom_bus of ide {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 04:06:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Found default for hw_disk_bus of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 04:06:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Found default for hw_input_bus of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 04:06:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Found default for hw_pointer_model of None {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 04:06:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Found default for hw_video_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 04:06:42 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Found default for hw_vif_model of virtio {{(pid=72081) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 12 04:06:42 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 04:06:42 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681272402.3391573, 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 04:06:42 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] VM Started (Lifecycle Event)
Apr 12 04:06:42 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 04:06:42 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 04:06:42 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 12 04:06:42 user nova-compute[72081]: INFO nova.compute.manager [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Took 5.48 seconds to spawn the instance on the hypervisor.
Apr 12 04:06:42 user nova-compute[72081]: DEBUG nova.compute.manager [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 04:06:42 user nova-compute[72081]: INFO nova.compute.manager [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Took 6.39 seconds to build instance.
Apr 12 04:06:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-bc722248-17da-4791-8637-fb935faf864a tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.489s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:06:42 user nova-compute[72081]: DEBUG nova.compute.manager [req-4e056d41-cc8d-40b6-96a6-3594163bafd7 req-e1e5e170-30b8-4b78-9b26-315bf8a046f1 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:06:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4e056d41-cc8d-40b6-96a6-3594163bafd7 req-e1e5e170-30b8-4b78-9b26-315bf8a046f1 service nova] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:06:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4e056d41-cc8d-40b6-96a6-3594163bafd7 req-e1e5e170-30b8-4b78-9b26-315bf8a046f1 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:06:42 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-4e056d41-cc8d-40b6-96a6-3594163bafd7 req-e1e5e170-30b8-4b78-9b26-315bf8a046f1 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:06:42 user nova-compute[72081]: DEBUG nova.compute.manager [req-4e056d41-cc8d-40b6-96a6-3594163bafd7 req-e1e5e170-30b8-4b78-9b26-315bf8a046f1 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] No waiting events found dispatching network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:06:42 user nova-compute[72081]: WARNING nova.compute.manager [req-4e056d41-cc8d-40b6-96a6-3594163bafd7 req-e1e5e170-30b8-4b78-9b26-315bf8a046f1 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received unexpected event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c for instance with vm_state active and task_state None.
Apr 12 04:06:42 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:06:42 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Cleaning up deleted instances with incomplete migration  {{(pid=72081) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 12 04:06:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:44 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:06:44 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:06:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 04:06:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Didn't find any instances for network info cache update. {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 12 04:06:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:53 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:06:53 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Cleaning up deleted instances {{(pid=72081) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 12 04:06:53 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] There are 0 instances to clean {{(pid=72081) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 12 04:06:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:06:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:07:02 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:07:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:07:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:07:05 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:07:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:07:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:07:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:07:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:07:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:07:28 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:07:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:07:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:07:30 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:07:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:07:34 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:07:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:07:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:07:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:07:34 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 04:07:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:07:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.140s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:07:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:07:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.142s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:07:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:07:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:07:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:07:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:07:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:07:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk --force-share --output=json" returned: 0 in 0.154s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:07:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:07:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:07:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:07:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:07:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:07:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:07:36 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:07:36 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:07:36 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8653MB free_disk=25.92886734008789GB free_vcpus=9 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 04:07:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:07:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:07:36 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance de40e137-84e3-4d85-895b-1f6da2673457 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:07:36 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 6cd92d70-a2af-4b2a-97c3-2981c157278d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:07:36 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:07:36 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 04:07:36 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=896MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 04:07:36 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 04:07:36 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 04:07:36 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 04:07:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.252s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:07:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:07:39 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:07:39 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:07:39 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:07:39 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 04:07:39 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:07:40 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:07:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:07:44 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_power_states {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:07:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Triggering sync for uuid de40e137-84e3-4d85-895b-1f6da2673457 {{(pid=72081) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 12 04:07:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Triggering sync for uuid 6cd92d70-a2af-4b2a-97c3-2981c157278d {{(pid=72081) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 12 04:07:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Triggering sync for uuid 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 {{(pid=72081) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 12 04:07:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "de40e137-84e3-4d85-895b-1f6da2673457" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:07:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "de40e137-84e3-4d85-895b-1f6da2673457" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:07:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "6cd92d70-a2af-4b2a-97c3-2981c157278d" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:07:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "6cd92d70-a2af-4b2a-97c3-2981c157278d" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:07:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:07:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:07:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "de40e137-84e3-4d85-895b-1f6da2673457" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.025s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:07:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "6cd92d70-a2af-4b2a-97c3-2981c157278d" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.027s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:07:44 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.029s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:07:45 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:07:45 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 04:07:45 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Rebuilding the list of instances to heal {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 12 04:07:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:07:45 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:07:45 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 04:07:45 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lazy-loading 'info_cache' on Instance uuid de40e137-84e3-4d85-895b-1f6da2673457 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:07:46 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Updating instance_info_cache with network_info: [{"id": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "address": "fa:16:3e:c4:e2:d7", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap16a3f234-3d", "ovs_interfaceid": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:07:46 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:07:46 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 04:07:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:07:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:07:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:08:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-2438f0bb-349c-484a-8e78-3182c0aee19b req-0b703007-6729-4be3-b22c-ddb11e4e1077 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-changed-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:08:26 user nova-compute[72081]: DEBUG nova.compute.manager [req-2438f0bb-349c-484a-8e78-3182c0aee19b req-0b703007-6729-4be3-b22c-ddb11e4e1077 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Refreshing instance network info cache due to event network-changed-b2759903-00b2-4ce7-8693-e93f30bb850c. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 04:08:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2438f0bb-349c-484a-8e78-3182c0aee19b req-0b703007-6729-4be3-b22c-ddb11e4e1077 service nova] Acquiring lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:08:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2438f0bb-349c-484a-8e78-3182c0aee19b req-0b703007-6729-4be3-b22c-ddb11e4e1077 service nova] Acquired lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:08:26 user nova-compute[72081]: DEBUG nova.network.neutron [req-2438f0bb-349c-484a-8e78-3182c0aee19b req-0b703007-6729-4be3-b22c-ddb11e4e1077 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Refreshing network info cache for port b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 04:08:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:08:26 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:08:26 user nova-compute[72081]: DEBUG nova.objects.instance [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'flavor' on Instance uuid 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.052s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:08:27 user nova-compute[72081]: INFO nova.compute.manager [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Attaching volume ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 to /dev/vdb
Apr 12 04:08:27 user nova-compute[72081]: DEBUG os_brick.utils [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[66211ad7-1cba-4c3c-8d78-62eb3981e3a5]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:c011b47bd53f\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[7cadfe9f-68b1-4f13-9642-83d12e193c12]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[8469f953-81ec-4d33-a265-e5b9b51eae3d]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[6378604f-36ba-4fed-a2f6-90f68a887eb3]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Running cmd (subprocess): nvme version {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] 'nvme version' failed. Not Retrying. {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.nvmeof [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] nvme not present on system {{(pid=72081) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 12 04:08:27 user nova-compute[72081]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 12 04:08:27 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[83e18eac-896c-4e6c-a540-7010005f0b8e]: (4, '') {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72081) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.lightos [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] LIGHTOS: no hostnqn found. {{(pid=72081) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG os_brick.utils [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] <== get_connector_properties: return (123ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:c011b47bd53f', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG nova.virt.block_device [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Updating existing volume attachment record: 24ad3f65-9df0-428c-b91f-5b69a59a6044 {{(pid=72081) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG nova.network.neutron [req-2438f0bb-349c-484a-8e78-3182c0aee19b req-0b703007-6729-4be3-b22c-ddb11e4e1077 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Updated VIF entry in instance network info cache for port b2759903-00b2-4ce7-8693-e93f30bb850c. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG nova.network.neutron [req-2438f0bb-349c-484a-8e78-3182c0aee19b req-0b703007-6729-4be3-b22c-ddb11e4e1077 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Updating instance_info_cache with network_info: [{"id": "b2759903-00b2-4ce7-8693-e93f30bb850c", "address": "fa:16:3e:b6:f9:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.251", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb2759903-00", "ovs_interfaceid": "b2759903-00b2-4ce7-8693-e93f30bb850c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:08:27 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2438f0bb-349c-484a-8e78-3182c0aee19b req-0b703007-6729-4be3-b22c-ddb11e4e1077 service nova] Releasing lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:08:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:30 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Calling os-brick to attach iSCSI Volume {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ksDFqytK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 04:08:31 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 04:08:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 04:08:31 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 04:08:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 04:08:31 user nova-compute[72081]: INFO os_brick.initiator.connectors.iscsi [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 -p 172.16.0.220:3260 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e325a0df-3068-43bf-8c93-21abbcfa3d80]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 12 04:08:31 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 -p 172.16.0.220:3260 --interface default --op new {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e3b5d43c-cffa-423d-b724-81d98ba8fae4]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64] added\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64] added
Apr 12 04:08:31 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[42b795b2-7550-4bba-a115-12dfbf2d14af]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.016s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[9ad3363a-4f47-4049-b906-ea1d3aa5d53b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v ksDFqytK {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v ksDFqytK" returned: 0 in 0.021s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6a3b871-bba2-41ab-8265-d3085d8b07e9]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'ksDFqytK'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.029s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c2f274b0-6261-477f-aa7a-eb463abab46b]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e3581b53-2120-4483-85ab-81a53b3d0d6f]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Apr 12 04:08:31 user nova-compute[72081]:  {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Apr 12 04:08:31 user nova-compute[72081]:  {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 04:08:31 user nova-compute[72081]: WARNING os_brick.initiator.connectors.iscsi [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Apr 12 04:08:31 user nova-compute[72081]: 
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 -p 172.16.0.220:3260 --login {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 -p 172.16.0.220:3260 --login" returned: 0 in 0.048s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bab93788-5ca6-4629-bf49-b6049f9a28ab]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64, portal: 172.16.0.220,3260]
Apr 12 04:08:31 user nova-compute[72081]: Login to [iface: default, target: iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64, portal: 172.16.0.220,3260] successful.
Apr 12 04:08:31 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.025s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[f5ea48c2-b921-43ba-91ac-672284b23327]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[5cee8ba0-e4d4-4b29-9749-fe55ae450f48]: (4, ('tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 (non-flash)
Apr 12 04:08:31 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] iscsi session list stdout=tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 (non-flash)
Apr 12 04:08:31 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.264s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Connected to 172.16.0.220:3260 {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] HCTL ('33', '-', '-', 0) found on session 54 with lun 0 {{(pid=72081) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72081) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[144ac3b2-617c-4bd5-a5f2-8226e856fc98]: (4, ('- - 0', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Searching for a device in session 54 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72081) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 12 04:08:31 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ksDFqytK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=72081) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 12 04:08:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.294s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 04:08:32 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] <== connect_volume: return (1296ms) {'type': 'block', 'scsi_wwn': '26538643037636430', 'path': '/dev/sdb'} {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 04:08:32 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26538643037636430', 'path': '/dev/sdb'} {{(pid=72081) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 12 04:08:33 user nova-compute[72081]: DEBUG nova.objects.instance [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'flavor' on Instance uuid 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:08:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] attach device xml: <disk type="block" device="disk">
Apr 12 04:08:33 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 04:08:33 user nova-compute[72081]:   <source dev="/dev/sdb"/>
Apr 12 04:08:33 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 04:08:33 user nova-compute[72081]:   <serial>ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64</serial>
Apr 12 04:08:33 user nova-compute[72081]: </disk>
Apr 12 04:08:33 user nova-compute[72081]:  {{(pid=72081) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 12 04:08:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 04:08:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 04:08:33 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] No VIF found with MAC fa:16:3e:b6:f9:3b, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 04:08:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-d7a7efda-f472-4439-8b01-3a6c788f88d2 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.340s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:08:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:33 user nova-compute[72081]: INFO nova.compute.manager [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Rescuing
Apr 12 04:08:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:08:33 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquired lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:08:33 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 04:08:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:34 user nova-compute[72081]: DEBUG nova.network.neutron [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Updating instance_info_cache with network_info: [{"id": "b2759903-00b2-4ce7-8693-e93f30bb850c", "address": "fa:16:3e:b6:f9:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.251", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb2759903-00", "ovs_interfaceid": "b2759903-00b2-4ce7-8693-e93f30bb850c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:08:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Releasing lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:08:34 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:08:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:08:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:08:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:08:34 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 04:08:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.compute.manager [req-238daf88-f003-4867-94b4-3bb93df02c15 req-88ae907f-cce4-4003-8ac0-84cf251a0a48 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-unplugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-238daf88-f003-4867-94b4-3bb93df02c15 req-88ae907f-cce4-4003-8ac0-84cf251a0a48 service nova] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-238daf88-f003-4867-94b4-3bb93df02c15 req-88ae907f-cce4-4003-8ac0-84cf251a0a48 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-238daf88-f003-4867-94b4-3bb93df02c15 req-88ae907f-cce4-4003-8ac0-84cf251a0a48 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.compute.manager [req-238daf88-f003-4867-94b4-3bb93df02c15 req-88ae907f-cce4-4003-8ac0-84cf251a0a48 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] No waiting events found dispatching network-vif-unplugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:08:35 user nova-compute[72081]: WARNING nova.compute.manager [req-238daf88-f003-4867-94b4-3bb93df02c15 req-88ae907f-cce4-4003-8ac0-84cf251a0a48 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received unexpected event network-vif-unplugged-b2759903-00b2-4ce7-8693-e93f30bb850c for instance with vm_state active and task_state rescuing.
Apr 12 04:08:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:35 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Instance destroyed successfully.
Apr 12 04:08:35 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Attempting rescue
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} {{(pid=72081) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4289}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Instance directory exists: not creating {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4694}}
Apr 12 04:08:35 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Creating image(s)
Apr 12 04:08:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "/opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "/opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "/opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.007s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.objects.instance [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'trusted_certs' on Instance uuid 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk.rescue {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/436a1a611c7731cbd94d56c4dfa86ba390d6d5df,backing_fmt=raw /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk.rescue" returned: 0 in 0.054s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "436a1a611c7731cbd94d56c4dfa86ba390d6d5df" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.194s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.objects.instance [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'migration_context' on Instance uuid 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Created local disks {{(pid=72081) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Start _get_guest_xml network_info=[{"id": "b2759903-00b2-4ce7-8693-e93f30bb850c", "address": "fa:16:3e:b6:f9:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.251", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "vif_mac": "fa:16:3e:b6:f9:3b"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb2759903-00", "ovs_interfaceid": "b2759903-00b2-4ce7-8693-e93f30bb850c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': 'e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7', 'kernel_id': '', 'ramdisk_id': ''} block_device_info=None {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.objects.instance [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'resources' on Instance uuid 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.141s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.objects.instance [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'numa_topology' on Instance uuid 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:08:35 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:08:35 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72081) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-12T03:27:26Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-12T03:26:21Z,direct_url=<?>,disk_format='qcow2',id=e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='e0a848f7aa0143d98afdd0e4e05fe87c',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-12T03:26:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Flavor limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Image limits 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Flavor pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Image pref 0:0:0 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72081) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Got 1 possible topologies {{(pid=72081) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.virt.hardware [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72081) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.objects.instance [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'vcpu_model' on Instance uuid 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T04:06:35Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-1315935319',display_name='tempest-ServerRescueNegativeTestJSON-server-1315935319',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-1315935319',id=47,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLXMs69H3aVwHvUXehqGeCn+vQJTwljOWf0Mp9khd00+UAOaU6NKxayjTMFZ3zU9NHkMqh5MJ76PIyP2BqxywFz+m9V79UNQ6CkqenIlufndQEJ4s9KnY6yNND9omt/Bww==',key_name='tempest-keypair-469921698',keypairs=<?>,launch_index=0,launched_at=2023-04-12T04:06:42Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='a6a8a4f5d59e479f9ca7b73a9e270c4b',ramdisk_id='',reservation_id='r-dat2ec2g',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-529226261',owner_user_name='tempest-ServerRescueNegativeTestJSON-529226261-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-04-12T04:06:42Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='44a0206f1323440db4f8411a3c3c088d',uuid=24cfaf66-a8ca-4dc3-8b83-1fe7de583f43,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "b2759903-00b2-4ce7-8693-e93f30bb850c", "address": "fa:16:3e:b6:f9:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.251", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "vif_mac": "fa:16:3e:b6:f9:3b"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb2759903-00", "ovs_interfaceid": "b2759903-00b2-4ce7-8693-e93f30bb850c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72081) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Converting VIF {"id": "b2759903-00b2-4ce7-8693-e93f30bb850c", "address": "fa:16:3e:b6:f9:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.251", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "vif_mac": "fa:16:3e:b6:f9:3b"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb2759903-00", "ovs_interfaceid": "b2759903-00b2-4ce7-8693-e93f30bb850c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:b6:f9:3b,bridge_name='br-int',has_traffic_filtering=True,id=b2759903-00b2-4ce7-8693-e93f30bb850c,network=Network(311bda05-075c-4bb3-9469-b5117dd33353),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb2759903-00') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG nova.objects.instance [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'pci_devices' on Instance uuid 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:08:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:36 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] End _get_guest_xml xml=<domain type="kvm">
Apr 12 04:08:36 user nova-compute[72081]:   <uuid>24cfaf66-a8ca-4dc3-8b83-1fe7de583f43</uuid>
Apr 12 04:08:36 user nova-compute[72081]:   <name>instance-0000002f</name>
Apr 12 04:08:36 user nova-compute[72081]:   <memory>131072</memory>
Apr 12 04:08:36 user nova-compute[72081]:   <vcpu>1</vcpu>
Apr 12 04:08:36 user nova-compute[72081]:   <metadata>
Apr 12 04:08:36 user nova-compute[72081]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 12 04:08:36 user nova-compute[72081]:       <nova:package version="0.0.0"/>
Apr 12 04:08:36 user nova-compute[72081]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-1315935319</nova:name>
Apr 12 04:08:36 user nova-compute[72081]:       <nova:creationTime>2023-04-12 04:08:35</nova:creationTime>
Apr 12 04:08:36 user nova-compute[72081]:       <nova:flavor name="m1.nano">
Apr 12 04:08:36 user nova-compute[72081]:         <nova:memory>128</nova:memory>
Apr 12 04:08:36 user nova-compute[72081]:         <nova:disk>1</nova:disk>
Apr 12 04:08:36 user nova-compute[72081]:         <nova:swap>0</nova:swap>
Apr 12 04:08:36 user nova-compute[72081]:         <nova:ephemeral>0</nova:ephemeral>
Apr 12 04:08:36 user nova-compute[72081]:         <nova:vcpus>1</nova:vcpus>
Apr 12 04:08:36 user nova-compute[72081]:       </nova:flavor>
Apr 12 04:08:36 user nova-compute[72081]:       <nova:owner>
Apr 12 04:08:36 user nova-compute[72081]:         <nova:user uuid="44a0206f1323440db4f8411a3c3c088d">tempest-ServerRescueNegativeTestJSON-529226261-project-member</nova:user>
Apr 12 04:08:36 user nova-compute[72081]:         <nova:project uuid="a6a8a4f5d59e479f9ca7b73a9e270c4b">tempest-ServerRescueNegativeTestJSON-529226261</nova:project>
Apr 12 04:08:36 user nova-compute[72081]:       </nova:owner>
Apr 12 04:08:36 user nova-compute[72081]:       <nova:root type="image" uuid="e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7"/>
Apr 12 04:08:36 user nova-compute[72081]:       <nova:ports>
Apr 12 04:08:36 user nova-compute[72081]:         <nova:port uuid="b2759903-00b2-4ce7-8693-e93f30bb850c">
Apr 12 04:08:36 user nova-compute[72081]:           <nova:ip type="fixed" address="10.0.0.11" ipVersion="4"/>
Apr 12 04:08:36 user nova-compute[72081]:         </nova:port>
Apr 12 04:08:36 user nova-compute[72081]:       </nova:ports>
Apr 12 04:08:36 user nova-compute[72081]:     </nova:instance>
Apr 12 04:08:36 user nova-compute[72081]:   </metadata>
Apr 12 04:08:36 user nova-compute[72081]:   <sysinfo type="smbios">
Apr 12 04:08:36 user nova-compute[72081]:     <system>
Apr 12 04:08:36 user nova-compute[72081]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 12 04:08:36 user nova-compute[72081]:       <entry name="product">OpenStack Nova</entry>
Apr 12 04:08:36 user nova-compute[72081]:       <entry name="version">0.0.0</entry>
Apr 12 04:08:36 user nova-compute[72081]:       <entry name="serial">24cfaf66-a8ca-4dc3-8b83-1fe7de583f43</entry>
Apr 12 04:08:36 user nova-compute[72081]:       <entry name="uuid">24cfaf66-a8ca-4dc3-8b83-1fe7de583f43</entry>
Apr 12 04:08:36 user nova-compute[72081]:       <entry name="family">Virtual Machine</entry>
Apr 12 04:08:36 user nova-compute[72081]:     </system>
Apr 12 04:08:36 user nova-compute[72081]:   </sysinfo>
Apr 12 04:08:36 user nova-compute[72081]:   <os>
Apr 12 04:08:36 user nova-compute[72081]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 12 04:08:36 user nova-compute[72081]:     <smbios mode="sysinfo"/>
Apr 12 04:08:36 user nova-compute[72081]:   </os>
Apr 12 04:08:36 user nova-compute[72081]:   <features>
Apr 12 04:08:36 user nova-compute[72081]:     <acpi/>
Apr 12 04:08:36 user nova-compute[72081]:     <apic/>
Apr 12 04:08:36 user nova-compute[72081]:     <vmcoreinfo/>
Apr 12 04:08:36 user nova-compute[72081]:   </features>
Apr 12 04:08:36 user nova-compute[72081]:   <clock offset="utc">
Apr 12 04:08:36 user nova-compute[72081]:     <timer name="pit" tickpolicy="delay"/>
Apr 12 04:08:36 user nova-compute[72081]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 12 04:08:36 user nova-compute[72081]:     <timer name="hpet" present="no"/>
Apr 12 04:08:36 user nova-compute[72081]:   </clock>
Apr 12 04:08:36 user nova-compute[72081]:   <cpu mode="custom" match="exact">
Apr 12 04:08:36 user nova-compute[72081]:     <model>Nehalem</model>
Apr 12 04:08:36 user nova-compute[72081]:     <topology sockets="1" cores="1" threads="1"/>
Apr 12 04:08:36 user nova-compute[72081]:   </cpu>
Apr 12 04:08:36 user nova-compute[72081]:   <devices>
Apr 12 04:08:36 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 04:08:36 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 04:08:36 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk.rescue"/>
Apr 12 04:08:36 user nova-compute[72081]:       <target dev="vda" bus="virtio"/>
Apr 12 04:08:36 user nova-compute[72081]:     </disk>
Apr 12 04:08:36 user nova-compute[72081]:     <disk type="file" device="disk">
Apr 12 04:08:36 user nova-compute[72081]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 12 04:08:36 user nova-compute[72081]:       <source file="/opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk"/>
Apr 12 04:08:36 user nova-compute[72081]:       <target dev="vdb" bus="virtio"/>
Apr 12 04:08:36 user nova-compute[72081]:     </disk>
Apr 12 04:08:36 user nova-compute[72081]:     <interface type="ethernet">
Apr 12 04:08:36 user nova-compute[72081]:       <mac address="fa:16:3e:b6:f9:3b"/>
Apr 12 04:08:36 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 04:08:36 user nova-compute[72081]:       <mtu size="1442"/>
Apr 12 04:08:36 user nova-compute[72081]:       <target dev="tapb2759903-00"/>
Apr 12 04:08:36 user nova-compute[72081]:     </interface>
Apr 12 04:08:36 user nova-compute[72081]:     <serial type="pty">
Apr 12 04:08:36 user nova-compute[72081]:       <log file="/opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/console.log" append="off"/>
Apr 12 04:08:36 user nova-compute[72081]:     </serial>
Apr 12 04:08:36 user nova-compute[72081]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 12 04:08:36 user nova-compute[72081]:     <video>
Apr 12 04:08:36 user nova-compute[72081]:       <model type="virtio"/>
Apr 12 04:08:36 user nova-compute[72081]:     </video>
Apr 12 04:08:36 user nova-compute[72081]:     <rng model="virtio">
Apr 12 04:08:36 user nova-compute[72081]:       <backend model="random">/dev/urandom</backend>
Apr 12 04:08:36 user nova-compute[72081]:     </rng>
Apr 12 04:08:36 user nova-compute[72081]:     <controller type="usb" index="0" model="none"/>
Apr 12 04:08:36 user nova-compute[72081]:     <memballoon model="virtio">
Apr 12 04:08:36 user nova-compute[72081]:       <stats period="10"/>
Apr 12 04:08:36 user nova-compute[72081]:     </memballoon>
Apr 12 04:08:36 user nova-compute[72081]:   </devices>
Apr 12 04:08:36 user nova-compute[72081]: </domain>
Apr 12 04:08:36 user nova-compute[72081]:  {{(pid=72081) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 12 04:08:36 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Instance destroyed successfully.
Apr 12 04:08:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] No BDM found with device name vda, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 04:08:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72081) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 12 04:08:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] No VIF found with MAC fa:16:3e:b6:f9:3b, not building metadata {{(pid=72081) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 12 04:08:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:08:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] No waiting events found dispatching network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:08:37 user nova-compute[72081]: WARNING nova.compute.manager [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received unexpected event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c for instance with vm_state active and task_state rescuing.
Apr 12 04:08:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] No waiting events found dispatching network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:08:37 user nova-compute[72081]: WARNING nova.compute.manager [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received unexpected event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c for instance with vm_state active and task_state rescuing.
Apr 12 04:08:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] No waiting events found dispatching network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:08:37 user nova-compute[72081]: WARNING nova.compute.manager [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received unexpected event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c for instance with vm_state active and task_state rescuing.
Apr 12 04:08:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-unplugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] No waiting events found dispatching network-vif-unplugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:08:37 user nova-compute[72081]: WARNING nova.compute.manager [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received unexpected event network-vif-unplugged-b2759903-00b2-4ce7-8693-e93f30bb850c for instance with vm_state active and task_state rescuing.
Apr 12 04:08:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG nova.compute.manager [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] No waiting events found dispatching network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:08:37 user nova-compute[72081]: WARNING nova.compute.manager [req-5f2e4c0f-d1de-4e9c-9b64-888da3f56e9b req-77dbab89-be29-45ab-8e63-376e92984c6c service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received unexpected event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c for instance with vm_state active and task_state rescuing.
Apr 12 04:08:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk.rescue --force-share --output=json" returned: 0 in 0.144s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:08:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk.rescue --force-share --output=json" returned: 0 in 0.144s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:08:38 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 04:08:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:08:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:08:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:08:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:38 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:08:38 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:08:38 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8703MB free_disk=25.927902221679688GB free_vcpus=9 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 04:08:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:08:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:08:38 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance de40e137-84e3-4d85-895b-1f6da2673457 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:08:38 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 6cd92d70-a2af-4b2a-97c3-2981c157278d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:08:38 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:08:38 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 04:08:38 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=896MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 04:08:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:39 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 04:08:39 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 04:08:39 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 04:08:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.251s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:08:39 user nova-compute[72081]: DEBUG nova.compute.manager [req-1513c191-7546-4888-91df-cc59457b02b5 req-faff7de5-0f98-4d1a-bf24-ddafec8ee335 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:08:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1513c191-7546-4888-91df-cc59457b02b5 req-faff7de5-0f98-4d1a-bf24-ddafec8ee335 service nova] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:08:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1513c191-7546-4888-91df-cc59457b02b5 req-faff7de5-0f98-4d1a-bf24-ddafec8ee335 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:08:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1513c191-7546-4888-91df-cc59457b02b5 req-faff7de5-0f98-4d1a-bf24-ddafec8ee335 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:08:39 user nova-compute[72081]: DEBUG nova.compute.manager [req-1513c191-7546-4888-91df-cc59457b02b5 req-faff7de5-0f98-4d1a-bf24-ddafec8ee335 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] No waiting events found dispatching network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:08:39 user nova-compute[72081]: WARNING nova.compute.manager [req-1513c191-7546-4888-91df-cc59457b02b5 req-faff7de5-0f98-4d1a-bf24-ddafec8ee335 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received unexpected event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c for instance with vm_state active and task_state rescuing.
Apr 12 04:08:39 user nova-compute[72081]: DEBUG nova.compute.manager [req-1513c191-7546-4888-91df-cc59457b02b5 req-faff7de5-0f98-4d1a-bf24-ddafec8ee335 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:08:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1513c191-7546-4888-91df-cc59457b02b5 req-faff7de5-0f98-4d1a-bf24-ddafec8ee335 service nova] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:08:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1513c191-7546-4888-91df-cc59457b02b5 req-faff7de5-0f98-4d1a-bf24-ddafec8ee335 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:08:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1513c191-7546-4888-91df-cc59457b02b5 req-faff7de5-0f98-4d1a-bf24-ddafec8ee335 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:08:39 user nova-compute[72081]: DEBUG nova.compute.manager [req-1513c191-7546-4888-91df-cc59457b02b5 req-faff7de5-0f98-4d1a-bf24-ddafec8ee335 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] No waiting events found dispatching network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:08:39 user nova-compute[72081]: WARNING nova.compute.manager [req-1513c191-7546-4888-91df-cc59457b02b5 req-faff7de5-0f98-4d1a-bf24-ddafec8ee335 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received unexpected event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c for instance with vm_state active and task_state rescuing.
Apr 12 04:08:39 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Removed pending event for 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 due to event {{(pid=72081) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 12 04:08:39 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681272519.524989, 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 04:08:39 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] VM Resumed (Lifecycle Event)
Apr 12 04:08:39 user nova-compute[72081]: DEBUG nova.compute.manager [None req-8702ab0d-84a5-42fe-9ca6-726750dda939 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 04:08:39 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 04:08:39 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 04:08:39 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] During sync_power_state the instance has a pending task (rescuing). Skip.
Apr 12 04:08:39 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681272519.5268743, 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 04:08:39 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] VM Started (Lifecycle Event)
Apr 12 04:08:39 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 04:08:39 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 04:08:43 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:08:43 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:08:43 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:08:43 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:08:43 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:08:43 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 04:08:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:45 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:08:47 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:08:47 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 04:08:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-6cd92d70-a2af-4b2a-97c3-2981c157278d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:08:47 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-6cd92d70-a2af-4b2a-97c3-2981c157278d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:08:47 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 04:08:48 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Updating instance_info_cache with network_info: [{"id": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "address": "fa:16:3e:c0:fc:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapee1cfaf9-15", "ovs_interfaceid": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:08:48 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-6cd92d70-a2af-4b2a-97c3-2981c157278d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:08:48 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 04:08:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:08:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:09:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:09:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:09:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:09:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:09:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:09:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:09:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:09:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:09:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:09:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:09:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:09:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:09:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:09:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:09:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:09:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:09:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:09:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:09:30 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:09:31 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:09:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:09:34 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:09:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:09:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:09:34 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:09:34 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 04:09:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:09:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:09:34 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:09:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:09:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:09:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:09:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:09:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:09:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:09:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk.rescue --force-share --output=json" returned: 0 in 0.140s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:09:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:09:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk.rescue --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:09:35 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 04:09:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:09:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:09:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:09:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:09:36 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:09:36 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:09:36 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8660MB free_disk=25.907608032226562GB free_vcpus=9 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 04:09:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:09:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:09:36 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance de40e137-84e3-4d85-895b-1f6da2673457 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:09:36 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 6cd92d70-a2af-4b2a-97c3-2981c157278d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:09:36 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:09:36 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 04:09:36 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=896MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 04:09:36 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 04:09:36 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 04:09:36 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 04:09:36 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.246s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:09:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:09:41 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:09:41 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:09:41 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 04:09:41 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:09:41 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:09:42 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:09:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:09:48 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:09:48 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 04:09:48 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:09:48 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:09:48 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 04:09:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:09:49 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Updating instance_info_cache with network_info: [{"id": "b2759903-00b2-4ce7-8693-e93f30bb850c", "address": "fa:16:3e:b6:f9:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.251", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb2759903-00", "ovs_interfaceid": "b2759903-00b2-4ce7-8693-e93f30bb850c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:09:49 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:09:49 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 04:09:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:09:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:09:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:10:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:10:25 user nova-compute[72081]: INFO nova.compute.manager [None req-10a577c4-9564-4e3d-9c94-51d54360e2ee tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Unrescuing
Apr 12 04:10:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-10a577c4-9564-4e3d-9c94-51d54360e2ee tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:10:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-10a577c4-9564-4e3d-9c94-51d54360e2ee tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquired lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:10:25 user nova-compute[72081]: DEBUG nova.network.neutron [None req-10a577c4-9564-4e3d-9c94-51d54360e2ee tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Building network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 12 04:10:25 user nova-compute[72081]: DEBUG nova.network.neutron [None req-10a577c4-9564-4e3d-9c94-51d54360e2ee tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Updating instance_info_cache with network_info: [{"id": "b2759903-00b2-4ce7-8693-e93f30bb850c", "address": "fa:16:3e:b6:f9:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.251", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb2759903-00", "ovs_interfaceid": "b2759903-00b2-4ce7-8693-e93f30bb850c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:10:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-10a577c4-9564-4e3d-9c94-51d54360e2ee tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Releasing lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:10:25 user nova-compute[72081]: DEBUG nova.objects.instance [None req-10a577c4-9564-4e3d-9c94-51d54360e2ee tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'flavor' on Instance uuid 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:10:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:25 user nova-compute[72081]: DEBUG nova.compute.manager [req-ee166af2-3da8-4425-b461-1d4d12b8ec6c req-58b54289-f28c-449f-a7d3-e08e9778ed6e service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-unplugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:10:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ee166af2-3da8-4425-b461-1d4d12b8ec6c req-58b54289-f28c-449f-a7d3-e08e9778ed6e service nova] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:10:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ee166af2-3da8-4425-b461-1d4d12b8ec6c req-58b54289-f28c-449f-a7d3-e08e9778ed6e service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:10:25 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ee166af2-3da8-4425-b461-1d4d12b8ec6c req-58b54289-f28c-449f-a7d3-e08e9778ed6e service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:10:25 user nova-compute[72081]: DEBUG nova.compute.manager [req-ee166af2-3da8-4425-b461-1d4d12b8ec6c req-58b54289-f28c-449f-a7d3-e08e9778ed6e service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] No waiting events found dispatching network-vif-unplugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:10:25 user nova-compute[72081]: WARNING nova.compute.manager [req-ee166af2-3da8-4425-b461-1d4d12b8ec6c req-58b54289-f28c-449f-a7d3-e08e9778ed6e service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received unexpected event network-vif-unplugged-b2759903-00b2-4ce7-8693-e93f30bb850c for instance with vm_state rescued and task_state unrescuing.
Apr 12 04:10:25 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:26 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Instance destroyed successfully.
Apr 12 04:10:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:26 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:27 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] No waiting events found dispatching network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:10:28 user nova-compute[72081]: WARNING nova.compute.manager [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received unexpected event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c for instance with vm_state rescued and task_state unrescuing.
Apr 12 04:10:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] No waiting events found dispatching network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:10:28 user nova-compute[72081]: WARNING nova.compute.manager [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received unexpected event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c for instance with vm_state rescued and task_state unrescuing.
Apr 12 04:10:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] No waiting events found dispatching network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:10:28 user nova-compute[72081]: WARNING nova.compute.manager [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received unexpected event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c for instance with vm_state rescued and task_state unrescuing.
Apr 12 04:10:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-unplugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] No waiting events found dispatching network-vif-unplugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:10:28 user nova-compute[72081]: WARNING nova.compute.manager [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received unexpected event network-vif-unplugged-b2759903-00b2-4ce7-8693-e93f30bb850c for instance with vm_state rescued and task_state unrescuing.
Apr 12 04:10:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] No waiting events found dispatching network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:10:28 user nova-compute[72081]: WARNING nova.compute.manager [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received unexpected event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c for instance with vm_state rescued and task_state unrescuing.
Apr 12 04:10:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG nova.compute.manager [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] No waiting events found dispatching network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:10:28 user nova-compute[72081]: WARNING nova.compute.manager [req-1b379c36-b315-4421-893b-78a1692b9da9 req-7d16e2ec-0072-4547-91a1-f8148d19f528 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received unexpected event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c for instance with vm_state rescued and task_state unrescuing.
Apr 12 04:10:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:29 user nova-compute[72081]: DEBUG nova.virt.libvirt.host [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Removed pending event for 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 due to event {{(pid=72081) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 12 04:10:29 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681272629.6566415, 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 => Resumed> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 04:10:29 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] VM Resumed (Lifecycle Event)
Apr 12 04:10:29 user nova-compute[72081]: DEBUG nova.compute.manager [None req-10a577c4-9564-4e3d-9c94-51d54360e2ee tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 04:10:29 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 04:10:29 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 04:10:29 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] During sync_power_state the instance has a pending task (unrescuing). Skip.
Apr 12 04:10:29 user nova-compute[72081]: DEBUG nova.virt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Emitting event <LifecycleEvent: 1681272629.6587, 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 => Started> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 04:10:29 user nova-compute[72081]: INFO nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] VM Started (Lifecycle Event)
Apr 12 04:10:29 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 04:10:29 user nova-compute[72081]: DEBUG nova.compute.manager [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=72081) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 12 04:10:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-5acbe973-eb3c-4874-af11-ae76b28df6c2 req-ba34c65f-92a9-4626-9eeb-a810df7c7ea0 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:10:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5acbe973-eb3c-4874-af11-ae76b28df6c2 req-ba34c65f-92a9-4626-9eeb-a810df7c7ea0 service nova] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:10:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5acbe973-eb3c-4874-af11-ae76b28df6c2 req-ba34c65f-92a9-4626-9eeb-a810df7c7ea0 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:10:30 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5acbe973-eb3c-4874-af11-ae76b28df6c2 req-ba34c65f-92a9-4626-9eeb-a810df7c7ea0 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:10:30 user nova-compute[72081]: DEBUG nova.compute.manager [req-5acbe973-eb3c-4874-af11-ae76b28df6c2 req-ba34c65f-92a9-4626-9eeb-a810df7c7ea0 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] No waiting events found dispatching network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:10:30 user nova-compute[72081]: WARNING nova.compute.manager [req-5acbe973-eb3c-4874-af11-ae76b28df6c2 req-ba34c65f-92a9-4626-9eeb-a810df7c7ea0 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received unexpected event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c for instance with vm_state active and task_state None.
Apr 12 04:10:30 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:10:31 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:10:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:10:35 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:10:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:10:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:10:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:10:35 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 04:10:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:10:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:10:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:10:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.145s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:10:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:10:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:10:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:10:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:10:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:10:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:10:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:10:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:10:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 04:10:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:10:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:10:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:10:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:10:37 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:10:37 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:10:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8623MB free_disk=25.92519760131836GB free_vcpus=9 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 04:10:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:10:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:10:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance de40e137-84e3-4d85-895b-1f6da2673457 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:10:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 6cd92d70-a2af-4b2a-97c3-2981c157278d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:10:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:10:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 04:10:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=896MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 04:10:37 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 04:10:37 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 04:10:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 04:10:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.260s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:10:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:42 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:10:42 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:10:42 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 04:10:42 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:10:42 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:10:42 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:10:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:47 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:10:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:50 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:10:50 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 04:10:50 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Rebuilding the list of instances to heal {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 12 04:10:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:10:50 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:10:50 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 04:10:50 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lazy-loading 'info_cache' on Instance uuid de40e137-84e3-4d85-895b-1f6da2673457 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:10:51 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Updating instance_info_cache with network_info: [{"id": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "address": "fa:16:3e:c4:e2:d7", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap16a3f234-3d", "ovs_interfaceid": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:10:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:10:51 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 04:10:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:10:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:11:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:11:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:11:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:11:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:11:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:11:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:11:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:11:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:11:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:11:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:11:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:11:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:11:31 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:11:31 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:11:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:11:35 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:11:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:11:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:11:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:11:35 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 04:11:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:11:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.142s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:11:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:11:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:11:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:11:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:11:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:11:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:11:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:11:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:11:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:11:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:11:36 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=72081) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 12 04:11:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:11:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:11:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:11:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:11:37 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:11:37 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:11:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8671MB free_disk=25.924362182617188GB free_vcpus=9 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 04:11:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:11:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:11:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance de40e137-84e3-4d85-895b-1f6da2673457 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:11:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 6cd92d70-a2af-4b2a-97c3-2981c157278d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:11:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:11:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 04:11:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=896MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 04:11:37 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Refreshing inventories for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 12 04:11:37 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Updating ProviderTree inventory for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 12 04:11:37 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Updating inventory in ProviderTree for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 12 04:11:37 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Refreshing aggregate associations for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6, aggregates: None {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 12 04:11:37 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Refreshing trait associations for resource provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6, traits: COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_USB,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_RESCUE_BFV,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE41,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_STORAGE_BUS_FDC,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_STORAGE_BUS_IDE,COMPUTE_TRUSTED_CERTS,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_GRAPHICS_MODEL_CIRRUS,HW_CPU_X86_SSSE3,HW_CPU_X86_SSE,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_EXTEND,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_STORAGE_BUS_SATA,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_GRAPHICS_MODEL_QXL,HW_CPU_X86_SSE42,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_ACCELERATORS,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,HW_CPU_X86_MMX,COMPUTE_NODE,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_DEVICE_TAGGING,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_VIOMMU_MODEL_AUTO {{(pid=72081) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 12 04:11:37 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 04:11:37 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 04:11:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 04:11:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.495s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:11:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:11:42 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:11:42 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:11:42 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:11:42 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 04:11:43 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:11:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:11:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:11:44 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:11:45 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:11:48 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:11:48 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Cleaning up deleted instances with incomplete migration  {{(pid=72081) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 12 04:11:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:11:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:11:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:11:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:11:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:11:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:11:51 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:11:51 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 04:11:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-6cd92d70-a2af-4b2a-97c3-2981c157278d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:11:51 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-6cd92d70-a2af-4b2a-97c3-2981c157278d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:11:51 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 04:11:52 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Updating instance_info_cache with network_info: [{"id": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "address": "fa:16:3e:c0:fc:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapee1cfaf9-15", "ovs_interfaceid": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:11:52 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-6cd92d70-a2af-4b2a-97c3-2981c157278d" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:11:52 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 04:11:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:11:54 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:11:54 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Cleaning up deleted instances {{(pid=72081) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 12 04:11:54 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] There are 0 instances to clean {{(pid=72081) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 12 04:11:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:12:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:12:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:12:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:12:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:12:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:12:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:12:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:12:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:12:16 user nova-compute[72081]: DEBUG nova.compute.manager [req-c1947206-171a-4bc8-98bc-e0d1385f3905 req-1e6f9faa-8431-4464-bfa3-ca8ce996e7ec service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-changed-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:12:16 user nova-compute[72081]: DEBUG nova.compute.manager [req-c1947206-171a-4bc8-98bc-e0d1385f3905 req-1e6f9faa-8431-4464-bfa3-ca8ce996e7ec service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Refreshing instance network info cache due to event network-changed-b2759903-00b2-4ce7-8693-e93f30bb850c. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 04:12:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c1947206-171a-4bc8-98bc-e0d1385f3905 req-1e6f9faa-8431-4464-bfa3-ca8ce996e7ec service nova] Acquiring lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:12:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c1947206-171a-4bc8-98bc-e0d1385f3905 req-1e6f9faa-8431-4464-bfa3-ca8ce996e7ec service nova] Acquired lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:12:16 user nova-compute[72081]: DEBUG nova.network.neutron [req-c1947206-171a-4bc8-98bc-e0d1385f3905 req-1e6f9faa-8431-4464-bfa3-ca8ce996e7ec service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Refreshing network info cache for port b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 04:12:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:12:16 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.004s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:12:16 user nova-compute[72081]: INFO nova.compute.manager [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Detaching volume ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64
Apr 12 04:12:17 user nova-compute[72081]: INFO nova.virt.block_device [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Attempting to driver detach volume ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 from mountpoint /dev/vdb
Apr 12 04:12:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Attempting to detach device vdb from instance 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 from the persistent domain config. {{(pid=72081) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 04:12:17 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 04:12:17 user nova-compute[72081]:   <source dev="/dev/sdb"/>
Apr 12 04:12:17 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 04:12:17 user nova-compute[72081]:   <serial>ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64</serial>
Apr 12 04:12:17 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 04:12:17 user nova-compute[72081]: </disk>
Apr 12 04:12:17 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 04:12:17 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Successfully detached device vdb from instance 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 from the persistent domain config.
Apr 12 04:12:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 from the live domain config. {{(pid=72081) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.guest [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] detach device xml: <disk type="block" device="disk">
Apr 12 04:12:17 user nova-compute[72081]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 12 04:12:17 user nova-compute[72081]:   <source dev="/dev/sdb"/>
Apr 12 04:12:17 user nova-compute[72081]:   <target dev="vdb" bus="virtio"/>
Apr 12 04:12:17 user nova-compute[72081]:   <serial>ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64</serial>
Apr 12 04:12:17 user nova-compute[72081]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 12 04:12:17 user nova-compute[72081]: </disk>
Apr 12 04:12:17 user nova-compute[72081]:  {{(pid=72081) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG nova.network.neutron [req-c1947206-171a-4bc8-98bc-e0d1385f3905 req-1e6f9faa-8431-4464-bfa3-ca8ce996e7ec service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Updated VIF entry in instance network info cache for port b2759903-00b2-4ce7-8693-e93f30bb850c. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG nova.network.neutron [req-c1947206-171a-4bc8-98bc-e0d1385f3905 req-1e6f9faa-8431-4464-bfa3-ca8ce996e7ec service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Updating instance_info_cache with network_info: [{"id": "b2759903-00b2-4ce7-8693-e93f30bb850c", "address": "fa:16:3e:b6:f9:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.251", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb2759903-00", "ovs_interfaceid": "b2759903-00b2-4ce7-8693-e93f30bb850c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-c1947206-171a-4bc8-98bc-e0d1385f3905 req-1e6f9faa-8431-4464-bfa3-ca8ce996e7ec service nova] Releasing lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-f9e6991d-7572-4eae-9e56-6ad8339c3772 None None] Received event <DeviceRemovedEvent: 1681272737.5665982, 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72081) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.driver [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 {{(pid=72081) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 12 04:12:17 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Successfully detached device vdb from instance 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 from the live domain config.
Apr 12 04:12:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] calling os-brick to detach iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f3f942806a0>, {'target_iqn': 'iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ksDFqytK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 12 04:12:17 user nova-compute[72081]: WARNING os_brick.initiator.connectors.base [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 12 04:12:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[457f3f05-f9d2-4518-8858-26fce2fea3ef]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 12 04:12:17 user nova-compute[72081]: No targets found.
Apr 12 04:12:17 user nova-compute[72081]: iSNS:
Apr 12 04:12:17 user nova-compute[72081]: No targets found.
Apr 12 04:12:17 user nova-compute[72081]: STATIC:
Apr 12 04:12:17 user nova-compute[72081]: Target: iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64
Apr 12 04:12:17 user nova-compute[72081]:         Portal: 172.16.0.220:3260,-1
Apr 12 04:12:17 user nova-compute[72081]:                 Iface Name: default
Apr 12 04:12:17 user nova-compute[72081]: FIRMWARE:
Apr 12 04:12:17 user nova-compute[72081]: No targets found.
Apr 12 04:12:17 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 12 04:12:17 user nova-compute[72081]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 12 04:12:17 user nova-compute[72081]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=72081) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64', 0)] {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[82cea446-47a6-4031-9584-da0c6ca8e14a]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[08dfad43-355e-4572-acad-a3713f58fe36]: (4, ('tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 (non-flash)\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 (non-flash)
Apr 12 04:12:17 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] iscsi session list stdout=tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 (non-flash)
Apr 12 04:12:17 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f3f744e2560>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64'): ({'sdb'}, set())}) {{(pid=72081) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Removing single pathed devices sdb {{(pid=72081) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e6777bed-9f4f-4a9e-b74d-31f5be26455b]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[de5c5b4f-a15e-46b4-900a-2d2f7a2b1d0a]: (4, ('ok\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Flushing IO for device /dev/sdb {{(pid=72081) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.015s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[c65f2da6-0c94-4550-9380-676b1d776d94]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72081) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.038s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e50f7b9a-ac12-48b2-a53e-5375d232811d]: (4, ('1', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG os_brick.initiator.linuxscsi [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] SCSI volumes sdb have been removed. {{(pid=72081) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64')] {{(pid=72081) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb92e402-bd85-43ca-b4ae-eccbc14b8c83]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 -p 172.16.0.220:3260 --logout {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 -p 172.16.0.220:3260 --logout" returned: 0 in 0.054s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[e6d7061a-4ad2-4218-886e-1ed27b657baf]: (4, ('Logging out of session [sid: 54, target: iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64, portal: 172.16.0.220,3260]\nLogout of [sid: 54, target: iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 54, target: iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64, portal: 172.16.0.220,3260]
Apr 12 04:12:17 user nova-compute[72081]: Logout of [sid: 54, target: iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64, portal: 172.16.0.220,3260] successful.
Apr 12 04:12:17 user nova-compute[72081]:  stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 -p 172.16.0.220:3260 --op delete {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ccbf6a5c-bbdb-4d91-bcb3-a9963485dd64 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=82083) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG oslo.privsep.daemon [-] privsep: reply[d484de92-c0e7-4711-a1e6-2d2cfcf2050e]: (4, ('', '')) {{(pid=82083) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72081) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.base [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.287s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG os_brick.initiator.connectors.iscsi [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] <== disconnect_volume: return (289ms) None {{(pid=72081) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 12 04:12:17 user nova-compute[72081]: DEBUG nova.virt.libvirt.volume.iscsi [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Disconnected iSCSI Volume {{(pid=72081) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 12 04:12:18 user nova-compute[72081]: DEBUG nova.objects.instance [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'flavor' on Instance uuid 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:12:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-cdf843f8-c3d3-474d-8b32-e5f8b115ca18 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.706s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:12:18 user nova-compute[72081]: DEBUG nova.compute.manager [req-5f0cba19-d416-4098-ad09-57984d1c340b req-8829306a-d98d-49c6-a80e-b585dcf93a05 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-changed-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:12:18 user nova-compute[72081]: DEBUG nova.compute.manager [req-5f0cba19-d416-4098-ad09-57984d1c340b req-8829306a-d98d-49c6-a80e-b585dcf93a05 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Refreshing instance network info cache due to event network-changed-b2759903-00b2-4ce7-8693-e93f30bb850c. {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 12 04:12:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f0cba19-d416-4098-ad09-57984d1c340b req-8829306a-d98d-49c6-a80e-b585dcf93a05 service nova] Acquiring lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:12:18 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f0cba19-d416-4098-ad09-57984d1c340b req-8829306a-d98d-49c6-a80e-b585dcf93a05 service nova] Acquired lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:12:18 user nova-compute[72081]: DEBUG nova.network.neutron [req-5f0cba19-d416-4098-ad09-57984d1c340b req-8829306a-d98d-49c6-a80e-b585dcf93a05 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Refreshing network info cache for port b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 12 04:12:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:12:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:12:19 user nova-compute[72081]: DEBUG nova.network.neutron [req-5f0cba19-d416-4098-ad09-57984d1c340b req-8829306a-d98d-49c6-a80e-b585dcf93a05 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Updated VIF entry in instance network info cache for port b2759903-00b2-4ce7-8693-e93f30bb850c. {{(pid=72081) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 12 04:12:19 user nova-compute[72081]: DEBUG nova.network.neutron [req-5f0cba19-d416-4098-ad09-57984d1c340b req-8829306a-d98d-49c6-a80e-b585dcf93a05 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Updating instance_info_cache with network_info: [{"id": "b2759903-00b2-4ce7-8693-e93f30bb850c", "address": "fa:16:3e:b6:f9:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.251", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb2759903-00", "ovs_interfaceid": "b2759903-00b2-4ce7-8693-e93f30bb850c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:12:19 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-5f0cba19-d416-4098-ad09-57984d1c340b req-8829306a-d98d-49c6-a80e-b585dcf93a05 service nova] Releasing lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:12:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:12:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:12:33 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:12:33 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:12:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:12:35 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:12:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:12:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:12:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:12:35 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 04:12:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:12:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:12:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:12:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:12:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:12:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:12:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:12:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:12:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:12:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:12:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:12:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:12:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:12:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:12:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:12:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:12:37 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:12:37 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:12:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8661MB free_disk=25.923599243164062GB free_vcpus=9 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 04:12:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:12:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:12:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance de40e137-84e3-4d85-895b-1f6da2673457 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:12:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 6cd92d70-a2af-4b2a-97c3-2981c157278d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:12:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:12:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 04:12:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=896MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 04:12:37 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 04:12:37 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 04:12:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 04:12:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.228s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:12:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:12:43 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:12:43 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:12:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:12:44 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:12:44 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:12:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 04:12:45 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:12:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:12:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:12:50 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:12:53 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:12:53 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 04:12:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:12:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:12:53 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 04:12:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:12:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:12:54 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Updating instance_info_cache with network_info: [{"id": "b2759903-00b2-4ce7-8693-e93f30bb850c", "address": "fa:16:3e:b6:f9:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.251", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb2759903-00", "ovs_interfaceid": "b2759903-00b2-4ce7-8693-e93f30bb850c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:12:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:12:54 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 04:12:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:13:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:13:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-30f1fd56-91b8-4d0a-adef-d4b0ed2a9453 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:13:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-30f1fd56-91b8-4d0a-adef-d4b0ed2a9453 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:13:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-30f1fd56-91b8-4d0a-adef-d4b0ed2a9453 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:13:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-30f1fd56-91b8-4d0a-adef-d4b0ed2a9453 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:13:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-30f1fd56-91b8-4d0a-adef-d4b0ed2a9453 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:13:07 user nova-compute[72081]: INFO nova.compute.manager [None req-30f1fd56-91b8-4d0a-adef-d4b0ed2a9453 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Terminating instance
Apr 12 04:13:07 user nova-compute[72081]: DEBUG nova.compute.manager [None req-30f1fd56-91b8-4d0a-adef-d4b0ed2a9453 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 04:13:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:13:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:13:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:13:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:13:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-34d3d376-64bf-4043-a7bb-54ffd873d69e req-51a9a703-041a-477a-afb1-a24c587794eb service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-unplugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:13:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-34d3d376-64bf-4043-a7bb-54ffd873d69e req-51a9a703-041a-477a-afb1-a24c587794eb service nova] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:13:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-34d3d376-64bf-4043-a7bb-54ffd873d69e req-51a9a703-041a-477a-afb1-a24c587794eb service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:13:07 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-34d3d376-64bf-4043-a7bb-54ffd873d69e req-51a9a703-041a-477a-afb1-a24c587794eb service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:13:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-34d3d376-64bf-4043-a7bb-54ffd873d69e req-51a9a703-041a-477a-afb1-a24c587794eb service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] No waiting events found dispatching network-vif-unplugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:13:07 user nova-compute[72081]: DEBUG nova.compute.manager [req-34d3d376-64bf-4043-a7bb-54ffd873d69e req-51a9a703-041a-477a-afb1-a24c587794eb service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-unplugged-b2759903-00b2-4ce7-8693-e93f30bb850c for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 04:13:07 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:13:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:13:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:13:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:13:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:13:08 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Instance destroyed successfully.
Apr 12 04:13:08 user nova-compute[72081]: DEBUG nova.objects.instance [None req-30f1fd56-91b8-4d0a-adef-d4b0ed2a9453 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'resources' on Instance uuid 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:13:08 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-30f1fd56-91b8-4d0a-adef-d4b0ed2a9453 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T04:06:35Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-1315935319',display_name='tempest-ServerRescueNegativeTestJSON-server-1315935319',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-1315935319',id=47,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLXMs69H3aVwHvUXehqGeCn+vQJTwljOWf0Mp9khd00+UAOaU6NKxayjTMFZ3zU9NHkMqh5MJ76PIyP2BqxywFz+m9V79UNQ6CkqenIlufndQEJ4s9KnY6yNND9omt/Bww==',key_name='tempest-keypair-469921698',keypairs=<?>,launch_index=0,launched_at=2023-04-12T04:08:39Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='a6a8a4f5d59e479f9ca7b73a9e270c4b',ramdisk_id='',reservation_id='r-dat2ec2g',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-529226261',owner_user_name='tempest-ServerRescueNegativeTestJSON-529226261-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T04:10:30Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='44a0206f1323440db4f8411a3c3c088d',uuid=24cfaf66-a8ca-4dc3-8b83-1fe7de583f43,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "b2759903-00b2-4ce7-8693-e93f30bb850c", "address": "fa:16:3e:b6:f9:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.251", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb2759903-00", "ovs_interfaceid": "b2759903-00b2-4ce7-8693-e93f30bb850c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 04:13:08 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-30f1fd56-91b8-4d0a-adef-d4b0ed2a9453 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Converting VIF {"id": "b2759903-00b2-4ce7-8693-e93f30bb850c", "address": "fa:16:3e:b6:f9:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.251", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb2759903-00", "ovs_interfaceid": "b2759903-00b2-4ce7-8693-e93f30bb850c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 04:13:08 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-30f1fd56-91b8-4d0a-adef-d4b0ed2a9453 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:b6:f9:3b,bridge_name='br-int',has_traffic_filtering=True,id=b2759903-00b2-4ce7-8693-e93f30bb850c,network=Network(311bda05-075c-4bb3-9469-b5117dd33353),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb2759903-00') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 04:13:08 user nova-compute[72081]: DEBUG os_vif [None req-30f1fd56-91b8-4d0a-adef-d4b0ed2a9453 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:b6:f9:3b,bridge_name='br-int',has_traffic_filtering=True,id=b2759903-00b2-4ce7-8693-e93f30bb850c,network=Network(311bda05-075c-4bb3-9469-b5117dd33353),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb2759903-00') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 04:13:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:13:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapb2759903-00, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 04:13:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:13:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:13:08 user nova-compute[72081]: INFO os_vif [None req-30f1fd56-91b8-4d0a-adef-d4b0ed2a9453 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:b6:f9:3b,bridge_name='br-int',has_traffic_filtering=True,id=b2759903-00b2-4ce7-8693-e93f30bb850c,network=Network(311bda05-075c-4bb3-9469-b5117dd33353),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb2759903-00')
Apr 12 04:13:08 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-30f1fd56-91b8-4d0a-adef-d4b0ed2a9453 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Deleting instance files /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43_del
Apr 12 04:13:08 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-30f1fd56-91b8-4d0a-adef-d4b0ed2a9453 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Deletion of /opt/stack/data/nova/instances/24cfaf66-a8ca-4dc3-8b83-1fe7de583f43_del complete
Apr 12 04:13:08 user nova-compute[72081]: INFO nova.compute.manager [None req-30f1fd56-91b8-4d0a-adef-d4b0ed2a9453 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Took 0.88 seconds to destroy the instance on the hypervisor.
Apr 12 04:13:08 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-30f1fd56-91b8-4d0a-adef-d4b0ed2a9453 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 04:13:08 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 04:13:08 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 04:13:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:13:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:13:09 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Took 1.09 seconds to deallocate network for instance.
Apr 12 04:13:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-30f1fd56-91b8-4d0a-adef-d4b0ed2a9453 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-30f1fd56-91b8-4d0a-adef-d4b0ed2a9453 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-30f1fd56-91b8-4d0a-adef-d4b0ed2a9453 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-30f1fd56-91b8-4d0a-adef-d4b0ed2a9453 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-30f1fd56-91b8-4d0a-adef-d4b0ed2a9453 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.150s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:13:09 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-30f1fd56-91b8-4d0a-adef-d4b0ed2a9453 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Deleted allocations for instance 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43
Apr 12 04:13:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-30f1fd56-91b8-4d0a-adef-d4b0ed2a9453 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.304s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] No waiting events found dispatching network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:13:09 user nova-compute[72081]: WARNING nova.compute.manager [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received unexpected event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c for instance with vm_state deleted and task_state None.
Apr 12 04:13:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] No waiting events found dispatching network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:13:09 user nova-compute[72081]: WARNING nova.compute.manager [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received unexpected event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c for instance with vm_state deleted and task_state None.
Apr 12 04:13:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] No waiting events found dispatching network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:13:09 user nova-compute[72081]: WARNING nova.compute.manager [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received unexpected event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c for instance with vm_state deleted and task_state None.
Apr 12 04:13:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-unplugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] No waiting events found dispatching network-vif-unplugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:13:09 user nova-compute[72081]: WARNING nova.compute.manager [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received unexpected event network-vif-unplugged-b2759903-00b2-4ce7-8693-e93f30bb850c for instance with vm_state deleted and task_state None.
Apr 12 04:13:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] Acquiring lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] Lock "24cfaf66-a8ca-4dc3-8b83-1fe7de583f43-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:13:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] No waiting events found dispatching network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:13:09 user nova-compute[72081]: WARNING nova.compute.manager [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received unexpected event network-vif-plugged-b2759903-00b2-4ce7-8693-e93f30bb850c for instance with vm_state deleted and task_state None.
Apr 12 04:13:09 user nova-compute[72081]: DEBUG nova.compute.manager [req-ca5a54de-b273-473f-93d4-dd155ac7a49d req-66542fc8-54c4-455b-878e-1cfa35ced382 service nova] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Received event network-vif-deleted-b2759903-00b2-4ce7-8693-e93f30bb850c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:13:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:13:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:13:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:13:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:13:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:13:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:13:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:13:23 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681272788.0870798, 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 04:13:23 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] VM Stopped (Lifecycle Event)
Apr 12 04:13:23 user nova-compute[72081]: DEBUG nova.compute.manager [None req-5fc55246-5f50-4446-baa1-917c682eb326 None None] [instance: 24cfaf66-a8ca-4dc3-8b83-1fe7de583f43] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 04:13:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:13:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:13:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:13:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:13:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:13:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:13:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:13:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:13:33 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:13:34 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:13:35 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:13:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:13:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:13:35 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:13:35 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 04:13:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:13:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.135s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:13:35 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:13:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk.rescue --force-share --output=json" returned: 0 in 0.146s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:13:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:13:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:13:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:13:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:13:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:13:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:13:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:13:36 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:13:37 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:13:37 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:13:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8728MB free_disk=25.94259262084961GB free_vcpus=10 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 04:13:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:13:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:13:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance de40e137-84e3-4d85-895b-1f6da2673457 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:13:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance 6cd92d70-a2af-4b2a-97c3-2981c157278d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:13:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 2 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 04:13:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=768MB phys_disk=40GB used_disk=2GB total_vcpus=12 used_vcpus=2 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 04:13:37 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 04:13:37 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 04:13:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 04:13:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.227s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:13:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:13:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:13:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:13:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:13:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:13:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:13:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:13:44 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:13:44 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:13:44 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:13:44 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 04:13:46 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:13:47 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:13:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:13:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:13:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:13:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:13:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:13:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:13:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:13:53 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:13:53 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 04:13:53 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Rebuilding the list of instances to heal {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 12 04:13:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 12 04:13:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquired lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 12 04:13:53 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Forcefully refreshing network info cache for instance {{(pid=72081) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 12 04:13:53 user nova-compute[72081]: DEBUG nova.objects.instance [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lazy-loading 'info_cache' on Instance uuid de40e137-84e3-4d85-895b-1f6da2673457 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:13:54 user nova-compute[72081]: DEBUG nova.network.neutron [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Updating instance_info_cache with network_info: [{"id": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "address": "fa:16:3e:c4:e2:d7", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap16a3f234-3d", "ovs_interfaceid": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:13:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Releasing lock "refresh_cache-de40e137-84e3-4d85-895b-1f6da2673457" {{(pid=72081) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 12 04:13:54 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Updated the network info_cache for instance {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 12 04:13:57 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:13:58 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:14:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:14:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a1863a4c-bcd7-4559-bb14-ff1e2e6b385d tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "6cd92d70-a2af-4b2a-97c3-2981c157278d" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:14:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a1863a4c-bcd7-4559-bb14-ff1e2e6b385d tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "6cd92d70-a2af-4b2a-97c3-2981c157278d" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:14:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a1863a4c-bcd7-4559-bb14-ff1e2e6b385d tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:14:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a1863a4c-bcd7-4559-bb14-ff1e2e6b385d tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:14:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a1863a4c-bcd7-4559-bb14-ff1e2e6b385d tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:14:03 user nova-compute[72081]: INFO nova.compute.manager [None req-a1863a4c-bcd7-4559-bb14-ff1e2e6b385d tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Terminating instance
Apr 12 04:14:03 user nova-compute[72081]: DEBUG nova.compute.manager [None req-a1863a4c-bcd7-4559-bb14-ff1e2e6b385d tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 04:14:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:14:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-2f2d40ec-3768-4bee-abb8-992102fdf6ac req-f8072d06-d27f-4c9f-8e5d-40cd7884d088 service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Received event network-vif-unplugged-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:14:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2f2d40ec-3768-4bee-abb8-992102fdf6ac req-f8072d06-d27f-4c9f-8e5d-40cd7884d088 service nova] Acquiring lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:14:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2f2d40ec-3768-4bee-abb8-992102fdf6ac req-f8072d06-d27f-4c9f-8e5d-40cd7884d088 service nova] Lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:14:03 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-2f2d40ec-3768-4bee-abb8-992102fdf6ac req-f8072d06-d27f-4c9f-8e5d-40cd7884d088 service nova] Lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:14:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-2f2d40ec-3768-4bee-abb8-992102fdf6ac req-f8072d06-d27f-4c9f-8e5d-40cd7884d088 service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] No waiting events found dispatching network-vif-unplugged-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:14:03 user nova-compute[72081]: DEBUG nova.compute.manager [req-2f2d40ec-3768-4bee-abb8-992102fdf6ac req-f8072d06-d27f-4c9f-8e5d-40cd7884d088 service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Received event network-vif-unplugged-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 04:14:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:14:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:14:03 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Instance destroyed successfully.
Apr 12 04:14:03 user nova-compute[72081]: DEBUG nova.objects.instance [None req-a1863a4c-bcd7-4559-bb14-ff1e2e6b385d tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'resources' on Instance uuid 6cd92d70-a2af-4b2a-97c3-2981c157278d {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:14:03 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-a1863a4c-bcd7-4559-bb14-ff1e2e6b385d tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:58:07Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-965303931',display_name='tempest-ServerRescueNegativeTestJSON-server-965303931',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-965303931',id=45,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-12T04:00:05Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='a6a8a4f5d59e479f9ca7b73a9e270c4b',ramdisk_id='',reservation_id='r-x2nt0nuo',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-529226261',owner_user_name='tempest-ServerRescueNegativeTestJSON-529226261-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T04:00:06Z,user_data=None,user_id='44a0206f1323440db4f8411a3c3c088d',uuid=6cd92d70-a2af-4b2a-97c3-2981c157278d,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='rescued') vif={"id": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "address": "fa:16:3e:c0:fc:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapee1cfaf9-15", "ovs_interfaceid": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 04:14:03 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-a1863a4c-bcd7-4559-bb14-ff1e2e6b385d tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Converting VIF {"id": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "address": "fa:16:3e:c0:fc:3b", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapee1cfaf9-15", "ovs_interfaceid": "ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 04:14:03 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-a1863a4c-bcd7-4559-bb14-ff1e2e6b385d tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:c0:fc:3b,bridge_name='br-int',has_traffic_filtering=True,id=ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda,network=Network(311bda05-075c-4bb3-9469-b5117dd33353),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapee1cfaf9-15') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 04:14:03 user nova-compute[72081]: DEBUG os_vif [None req-a1863a4c-bcd7-4559-bb14-ff1e2e6b385d tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:c0:fc:3b,bridge_name='br-int',has_traffic_filtering=True,id=ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda,network=Network(311bda05-075c-4bb3-9469-b5117dd33353),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapee1cfaf9-15') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 04:14:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:14:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapee1cfaf9-15, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 04:14:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:14:03 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:14:03 user nova-compute[72081]: INFO os_vif [None req-a1863a4c-bcd7-4559-bb14-ff1e2e6b385d tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:c0:fc:3b,bridge_name='br-int',has_traffic_filtering=True,id=ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda,network=Network(311bda05-075c-4bb3-9469-b5117dd33353),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapee1cfaf9-15')
Apr 12 04:14:03 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-a1863a4c-bcd7-4559-bb14-ff1e2e6b385d tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Deleting instance files /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d_del
Apr 12 04:14:03 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-a1863a4c-bcd7-4559-bb14-ff1e2e6b385d tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Deletion of /opt/stack/data/nova/instances/6cd92d70-a2af-4b2a-97c3-2981c157278d_del complete
Apr 12 04:14:03 user nova-compute[72081]: INFO nova.compute.manager [None req-a1863a4c-bcd7-4559-bb14-ff1e2e6b385d tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Took 0.68 seconds to destroy the instance on the hypervisor.
Apr 12 04:14:03 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-a1863a4c-bcd7-4559-bb14-ff1e2e6b385d tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 04:14:03 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 04:14:03 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 04:14:04 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:14:04 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Took 0.61 seconds to deallocate network for instance.
Apr 12 04:14:04 user nova-compute[72081]: DEBUG nova.compute.manager [req-d8c65335-4560-4100-8f9b-2289c23823dc req-7db9a1c4-a64d-489b-b3e3-b600132bd474 service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Received event network-vif-deleted-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:14:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a1863a4c-bcd7-4559-bb14-ff1e2e6b385d tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:14:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a1863a4c-bcd7-4559-bb14-ff1e2e6b385d tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:14:04 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-a1863a4c-bcd7-4559-bb14-ff1e2e6b385d tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 04:14:04 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-a1863a4c-bcd7-4559-bb14-ff1e2e6b385d tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 04:14:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a1863a4c-bcd7-4559-bb14-ff1e2e6b385d tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.141s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:14:04 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-a1863a4c-bcd7-4559-bb14-ff1e2e6b385d tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Deleted allocations for instance 6cd92d70-a2af-4b2a-97c3-2981c157278d
Apr 12 04:14:04 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-a1863a4c-bcd7-4559-bb14-ff1e2e6b385d tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "6cd92d70-a2af-4b2a-97c3-2981c157278d" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.606s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:14:05 user nova-compute[72081]: DEBUG nova.compute.manager [req-f141238b-8b55-4717-a161-887981b49bc3 req-4ee0df1f-8045-4873-a002-7b966e60c2f4 service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Received event network-vif-plugged-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:14:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f141238b-8b55-4717-a161-887981b49bc3 req-4ee0df1f-8045-4873-a002-7b966e60c2f4 service nova] Acquiring lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:14:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f141238b-8b55-4717-a161-887981b49bc3 req-4ee0df1f-8045-4873-a002-7b966e60c2f4 service nova] Lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:14:05 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-f141238b-8b55-4717-a161-887981b49bc3 req-4ee0df1f-8045-4873-a002-7b966e60c2f4 service nova] Lock "6cd92d70-a2af-4b2a-97c3-2981c157278d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:14:05 user nova-compute[72081]: DEBUG nova.compute.manager [req-f141238b-8b55-4717-a161-887981b49bc3 req-4ee0df1f-8045-4873-a002-7b966e60c2f4 service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] No waiting events found dispatching network-vif-plugged-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:14:05 user nova-compute[72081]: WARNING nova.compute.manager [req-f141238b-8b55-4717-a161-887981b49bc3 req-4ee0df1f-8045-4873-a002-7b966e60c2f4 service nova] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Received unexpected event network-vif-plugged-ee1cfaf9-15e5-43a6-854d-4fd6f4b59bda for instance with vm_state deleted and task_state None.
Apr 12 04:14:08 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:14:13 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:14:18 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681272843.7823417, 6cd92d70-a2af-4b2a-97c3-2981c157278d => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 04:14:18 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] VM Stopped (Lifecycle Event)
Apr 12 04:14:18 user nova-compute[72081]: DEBUG nova.compute.manager [None req-2f9cd145-2469-4930-9fab-b71e2aed2326 None None] [instance: 6cd92d70-a2af-4b2a-97c3-2981c157278d] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 04:14:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:14:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:14:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:14:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:14:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:14:18 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:14:23 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:14:28 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:14:33 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:14:34 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:14:36 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:14:37 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:14:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:14:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:14:37 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:14:37 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 04:14:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:14:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:14:37 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 12 04:14:38 user nova-compute[72081]: DEBUG oslo_concurrency.processutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72081) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 12 04:14:38 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:14:38 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:14:38 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8801MB free_disk=25.98037338256836GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 04:14:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:14:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:14:38 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Instance de40e137-84e3-4d85-895b-1f6da2673457 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=72081) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 12 04:14:38 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 04:14:38 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=640MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 04:14:38 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 04:14:38 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 04:14:38 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 04:14:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.213s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:14:38 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:14:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:14:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:14:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:14:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:14:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:14:43 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:14:44 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:14:45 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:14:46 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:14:46 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 04:14:47 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:14:48 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:14:49 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:14:53 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:14:53 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 04:14:53 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Didn't find any instances for network info cache update. {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 12 04:14:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-71a57010-2728-4a0b-b094-2ece3625e2d4 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "de40e137-84e3-4d85-895b-1f6da2673457" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:14:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-71a57010-2728-4a0b-b094-2ece3625e2d4 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "de40e137-84e3-4d85-895b-1f6da2673457" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:14:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-71a57010-2728-4a0b-b094-2ece3625e2d4 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "de40e137-84e3-4d85-895b-1f6da2673457-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:14:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-71a57010-2728-4a0b-b094-2ece3625e2d4 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:14:53 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-71a57010-2728-4a0b-b094-2ece3625e2d4 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:14:53 user nova-compute[72081]: INFO nova.compute.manager [None req-71a57010-2728-4a0b-b094-2ece3625e2d4 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Terminating instance
Apr 12 04:14:53 user nova-compute[72081]: DEBUG nova.compute.manager [None req-71a57010-2728-4a0b-b094-2ece3625e2d4 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Start destroying the instance on the hypervisor. {{(pid=72081) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 12 04:14:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:14:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:14:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:14:53 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:14:54 user nova-compute[72081]: DEBUG nova.compute.manager [req-b34cb142-6513-4d5c-9f96-1ba9d5eea10d req-9a00826a-fb7a-4c28-b372-49309522a8d4 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received event network-vif-unplugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:14:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b34cb142-6513-4d5c-9f96-1ba9d5eea10d req-9a00826a-fb7a-4c28-b372-49309522a8d4 service nova] Acquiring lock "de40e137-84e3-4d85-895b-1f6da2673457-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:14:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b34cb142-6513-4d5c-9f96-1ba9d5eea10d req-9a00826a-fb7a-4c28-b372-49309522a8d4 service nova] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.005s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:14:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-b34cb142-6513-4d5c-9f96-1ba9d5eea10d req-9a00826a-fb7a-4c28-b372-49309522a8d4 service nova] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:14:54 user nova-compute[72081]: DEBUG nova.compute.manager [req-b34cb142-6513-4d5c-9f96-1ba9d5eea10d req-9a00826a-fb7a-4c28-b372-49309522a8d4 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] No waiting events found dispatching network-vif-unplugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:14:54 user nova-compute[72081]: DEBUG nova.compute.manager [req-b34cb142-6513-4d5c-9f96-1ba9d5eea10d req-9a00826a-fb7a-4c28-b372-49309522a8d4 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received event network-vif-unplugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c for instance with task_state deleting. {{(pid=72081) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 12 04:14:54 user nova-compute[72081]: INFO nova.virt.libvirt.driver [-] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Instance destroyed successfully.
Apr 12 04:14:54 user nova-compute[72081]: DEBUG nova.objects.instance [None req-71a57010-2728-4a0b-b094-2ece3625e2d4 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lazy-loading 'resources' on Instance uuid de40e137-84e3-4d85-895b-1f6da2673457 {{(pid=72081) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 12 04:14:54 user nova-compute[72081]: DEBUG nova.virt.libvirt.vif [None req-71a57010-2728-4a0b-b094-2ece3625e2d4 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-12T03:58:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-1232389953',display_name='tempest-ServerRescueNegativeTestJSON-server-1232389953',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-1232389953',id=44,image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-12T04:02:55Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='a6a8a4f5d59e479f9ca7b73a9e270c4b',ramdisk_id='',reservation_id='r-tsy8s5du',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='e93a44d0-4b8a-47a9-bc0e-cefc41cd7ee7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-529226261',owner_user_name='tempest-ServerRescueNegativeTestJSON-529226261-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-12T04:04:46Z,user_data=None,user_id='44a0206f1323440db4f8411a3c3c088d',uuid=de40e137-84e3-4d85-895b-1f6da2673457,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "address": "fa:16:3e:c4:e2:d7", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap16a3f234-3d", "ovs_interfaceid": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 12 04:14:54 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-71a57010-2728-4a0b-b094-2ece3625e2d4 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Converting VIF {"id": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "address": "fa:16:3e:c4:e2:d7", "network": {"id": "311bda05-075c-4bb3-9469-b5117dd33353", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-1824903343-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a6a8a4f5d59e479f9ca7b73a9e270c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap16a3f234-3d", "ovs_interfaceid": "16a3f234-3d0d-446c-b6a0-0a240e35aa6c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 12 04:14:54 user nova-compute[72081]: DEBUG nova.network.os_vif_util [None req-71a57010-2728-4a0b-b094-2ece3625e2d4 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:c4:e2:d7,bridge_name='br-int',has_traffic_filtering=True,id=16a3f234-3d0d-446c-b6a0-0a240e35aa6c,network=Network(311bda05-075c-4bb3-9469-b5117dd33353),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap16a3f234-3d') {{(pid=72081) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 12 04:14:54 user nova-compute[72081]: DEBUG os_vif [None req-71a57010-2728-4a0b-b094-2ece3625e2d4 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:c4:e2:d7,bridge_name='br-int',has_traffic_filtering=True,id=16a3f234-3d0d-446c-b6a0-0a240e35aa6c,network=Network(311bda05-075c-4bb3-9469-b5117dd33353),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap16a3f234-3d') {{(pid=72081) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 12 04:14:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 24 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:14:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap16a3f234-3d, bridge=br-int, if_exists=True) {{(pid=72081) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 12 04:14:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:14:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:14:54 user nova-compute[72081]: INFO os_vif [None req-71a57010-2728-4a0b-b094-2ece3625e2d4 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:c4:e2:d7,bridge_name='br-int',has_traffic_filtering=True,id=16a3f234-3d0d-446c-b6a0-0a240e35aa6c,network=Network(311bda05-075c-4bb3-9469-b5117dd33353),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap16a3f234-3d')
Apr 12 04:14:54 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-71a57010-2728-4a0b-b094-2ece3625e2d4 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Deleting instance files /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457_del
Apr 12 04:14:54 user nova-compute[72081]: INFO nova.virt.libvirt.driver [None req-71a57010-2728-4a0b-b094-2ece3625e2d4 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Deletion of /opt/stack/data/nova/instances/de40e137-84e3-4d85-895b-1f6da2673457_del complete
Apr 12 04:14:54 user nova-compute[72081]: INFO nova.compute.manager [None req-71a57010-2728-4a0b-b094-2ece3625e2d4 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Took 0.69 seconds to destroy the instance on the hypervisor.
Apr 12 04:14:54 user nova-compute[72081]: DEBUG oslo.service.loopingcall [None req-71a57010-2728-4a0b-b094-2ece3625e2d4 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72081) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 12 04:14:54 user nova-compute[72081]: DEBUG nova.compute.manager [-] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Deallocating network for instance {{(pid=72081) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 12 04:14:54 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: de40e137-84e3-4d85-895b-1f6da2673457] deallocate_for_instance() {{(pid=72081) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 12 04:14:54 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:14:54 user nova-compute[72081]: DEBUG nova.network.neutron [-] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:14:54 user nova-compute[72081]: DEBUG nova.compute.manager [req-c5d24006-104c-4013-8cd8-1c5236fa287b req-22df1883-b05a-465e-83fd-f40d9f068cd7 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received event network-vif-deleted-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:14:54 user nova-compute[72081]: INFO nova.compute.manager [req-c5d24006-104c-4013-8cd8-1c5236fa287b req-22df1883-b05a-465e-83fd-f40d9f068cd7 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Neutron deleted interface 16a3f234-3d0d-446c-b6a0-0a240e35aa6c; detaching it from the instance and deleting it from the info cache
Apr 12 04:14:54 user nova-compute[72081]: DEBUG nova.network.neutron [req-c5d24006-104c-4013-8cd8-1c5236fa287b req-22df1883-b05a-465e-83fd-f40d9f068cd7 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Updating instance_info_cache with network_info: [] {{(pid=72081) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 12 04:14:54 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Took 0.47 seconds to deallocate network for instance.
Apr 12 04:14:54 user nova-compute[72081]: DEBUG nova.compute.manager [req-c5d24006-104c-4013-8cd8-1c5236fa287b req-22df1883-b05a-465e-83fd-f40d9f068cd7 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Detach interface failed, port_id=16a3f234-3d0d-446c-b6a0-0a240e35aa6c, reason: Instance de40e137-84e3-4d85-895b-1f6da2673457 could not be found. {{(pid=72081) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 12 04:14:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-71a57010-2728-4a0b-b094-2ece3625e2d4 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:14:54 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-71a57010-2728-4a0b-b094-2ece3625e2d4 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:14:55 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-71a57010-2728-4a0b-b094-2ece3625e2d4 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 04:14:55 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-71a57010-2728-4a0b-b094-2ece3625e2d4 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 04:14:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-71a57010-2728-4a0b-b094-2ece3625e2d4 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.108s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:14:55 user nova-compute[72081]: INFO nova.scheduler.client.report [None req-71a57010-2728-4a0b-b094-2ece3625e2d4 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Deleted allocations for instance de40e137-84e3-4d85-895b-1f6da2673457
Apr 12 04:14:55 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-71a57010-2728-4a0b-b094-2ece3625e2d4 tempest-ServerRescueNegativeTestJSON-529226261 tempest-ServerRescueNegativeTestJSON-529226261-project-member] Lock "de40e137-84e3-4d85-895b-1f6da2673457" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.459s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:14:56 user nova-compute[72081]: DEBUG nova.compute.manager [req-17ee8aeb-3bcf-4379-93ce-99584c9e902e req-b473f44d-c3dc-4566-85a3-9ed9bddac108 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received event network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 12 04:14:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-17ee8aeb-3bcf-4379-93ce-99584c9e902e req-b473f44d-c3dc-4566-85a3-9ed9bddac108 service nova] Acquiring lock "de40e137-84e3-4d85-895b-1f6da2673457-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:14:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-17ee8aeb-3bcf-4379-93ce-99584c9e902e req-b473f44d-c3dc-4566-85a3-9ed9bddac108 service nova] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:14:56 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [req-17ee8aeb-3bcf-4379-93ce-99584c9e902e req-b473f44d-c3dc-4566-85a3-9ed9bddac108 service nova] Lock "de40e137-84e3-4d85-895b-1f6da2673457-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:14:56 user nova-compute[72081]: DEBUG nova.compute.manager [req-17ee8aeb-3bcf-4379-93ce-99584c9e902e req-b473f44d-c3dc-4566-85a3-9ed9bddac108 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] No waiting events found dispatching network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c {{(pid=72081) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 12 04:14:56 user nova-compute[72081]: WARNING nova.compute.manager [req-17ee8aeb-3bcf-4379-93ce-99584c9e902e req-b473f44d-c3dc-4566-85a3-9ed9bddac108 service nova] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Received unexpected event network-vif-plugged-16a3f234-3d0d-446c-b6a0-0a240e35aa6c for instance with vm_state deleted and task_state None.
Apr 12 04:14:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:15:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:15:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:15:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:15:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:15:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:15:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:15:09 user nova-compute[72081]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681272894.297652, de40e137-84e3-4d85-895b-1f6da2673457 => Stopped> {{(pid=72081) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 12 04:15:09 user nova-compute[72081]: INFO nova.compute.manager [-] [instance: de40e137-84e3-4d85-895b-1f6da2673457] VM Stopped (Lifecycle Event)
Apr 12 04:15:09 user nova-compute[72081]: DEBUG nova.compute.manager [None req-e95d0b89-1bea-4fed-8756-71a14e97e7fe None None] [instance: de40e137-84e3-4d85-895b-1f6da2673457] Checking state {{(pid=72081) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 12 04:15:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:15:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:15:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:15:24 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:15:29 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:15:34 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:15:34 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:15:38 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:15:38 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:15:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:15:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:15:38 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:15:38 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Auditing locally available compute resources for user (node: user) {{(pid=72081) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 12 04:15:39 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:15:39 user nova-compute[72081]: WARNING nova.virt.libvirt.driver [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 12 04:15:39 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Hypervisor/Node resource view: name=user free_ram=8916MB free_disk=25.999515533447266GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72081) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 12 04:15:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 12 04:15:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 12 04:15:39 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 12 04:15:39 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=72081) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 12 04:15:39 user nova-compute[72081]: DEBUG nova.compute.provider_tree [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed in ProviderTree for provider: 2203ce45-27b6-4606-b33a-0f78fa74c5d6 {{(pid=72081) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 12 04:15:39 user nova-compute[72081]: DEBUG nova.scheduler.client.report [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Inventory has not changed for provider 2203ce45-27b6-4606-b33a-0f78fa74c5d6 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72081) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 12 04:15:39 user nova-compute[72081]: DEBUG nova.compute.resource_tracker [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Compute_service record updated for user:user {{(pid=72081) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 12 04:15:39 user nova-compute[72081]: DEBUG oslo_concurrency.lockutils [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.145s {{(pid=72081) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 12 04:15:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:15:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:15:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:15:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:15:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:15:39 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:15:44 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:15:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:15:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:15:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:15:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:15:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:15:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:15:44 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:15:46 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:15:46 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:15:48 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:15:48 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72081) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 12 04:15:49 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:15:49 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:15:50 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:15:54 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:15:55 user nova-compute[72081]: DEBUG oslo_service.periodic_task [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72081) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 12 04:15:55 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Starting heal instance info cache {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 12 04:15:55 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Rebuilding the list of instances to heal {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 12 04:15:55 user nova-compute[72081]: DEBUG nova.compute.manager [None req-9e71f176-fbcc-4316-887c-4ca3e2721f00 None None] Didn't find any instances for network info cache update. {{(pid=72081) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 12 04:15:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:15:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:15:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:15:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:15:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:15:59 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:16:04 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:16:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:16:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:16:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72081) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 12 04:16:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:16:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72081) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 12 04:16:09 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 26 {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 12 04:16:14 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 12 04:16:19 user nova-compute[72081]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72081) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}